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ABSTRACT 

In February* 1988* random samples of students from 
five countries (Ireland, Korea, Spain, the United Kingdom, and the 
United States) and four Canadian provinces (British Columbia, New 
Brunswick, Ontario, and Quebec) were given an assessment of 
mathematics and science achievement, and a questionnaire. This 
document summarizes the results of these tests and compares 
achievement, experiences and attitudes across the international 
sample. Findings are organized by achievement, instruction and 
attitude, and topics for both science and mathematics. The context 
the results for each country is summarized in a commentary. 
Appendices include procedures and data. (CW) 
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The Stud" m Brief 



Five countries and four Canadian provinces participated, (Canada 
does not have a federal system of education). A total of 12 student 
populations were included from: 



British Columbia 

Ireland 

Korea 

New Brunswick 
(English) 



New Brunswick 
(French) 

Ontario (English) 

Ontario (French) 

Quebec (English) 



Quebec (French) 
Spain 

United Kingdom* 
United States 



A£e Croup: Students were 13 years old (born January 1, 1974 — 
December 31, 1974), and were selected from public and private elemen- 
tary, middle, and secondary schools- 



Samples: A random sample of about 2,000 students from 100 differ- 
ent schools was selected from each population. In the United States, the 
sample size was about LO00 students in 200 schools. A total of 24000 
students was assessed. 

Assessment: Students were administered a 45-minute mathematics 
assessment (63 questions) and a 45*minute science assessment (60 
questions), selected from the total pool of 281 mathematics and 188 
science questions used in the 1986 United States 1 National Assessment 
of Educational Progress (NAEP). In addition, students answered ques- 
tions about their school experiences and attitudes, and teachers rated 
students exposure to the concepts tested by the items. 

Procedures: AN countries and provinces followed standardized 
administration procedures and administered the assessments during 
February 1988, 



•Thu United KirulddTn 5,im[% was drawn fenm student* in tfntiUuid. SccrfbniL and 
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I FOREWORD 



-A^n important part of the mission of Educational Testing Service (ETS) is improving 
the measurement of educational achievement. Since its creation more than AO years ago, 
ETS has invited scholars from all over the world to share the results; of our research, to 
learn about our experiments with new measurement technology, and to contribute their 
critiques, suggestions, and ideas to our endeavors Distinguished guests from many 
countries have graced our Princeton campus eveiy year $jnce 1947 and have become our 
colleagues. 

The first international Assessment of Educational Progress, reported in the pages that 
follow, takes this partnership one step further We have proceeded from the research 
laboratory and the classroom to the demonstration project. Working with colleagues 
from five other countries, ETSs measurement specialists have translated and adapted the 
techniques perfected in the United States by the National Assessment of Educational 
Progress (NAEPK and together they have conducted n .ini-assessments in five different 
countries. Achievement results that permit comparisons and present valid and reliable 
findings are contained in this report, 

The greater benefit in the minds of many, however, has been the opportunity each 
participating country has had to experiment with new measurement practices. 

The cast-effectiveness of sampling techniques, the power and the limitations of Item 
Response Theory, and the usefulness of new reporting techniques have ail been demon- 
strated, and their value in these various environments can now be judged more clearly. 
Experts in each of these countries have had hands on experiences with the problems and 
the potential of these new assessment techniques, 

ETS staff have benefited greatly from this experience and have been stimulated by the 
enthusiasm and the ideas of our international collaborators. We hope that this exercise 
has contributed in some small way to the broader understanding of how effective 
measurement can help to improve educational opportunity for all children. 

Gregory Ait rig 

President 

Educational Testing Service 
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INTRODUCTION 



The title of this report, ^4 World of Differences, suggests that from one culture to 
another, differing average levels of student achievement reflect differing aspirations with 
regard to education. Each culture has its idiosyncratic set of values and goes through 
cycles in which certain aspects of its schools* curricula are considered more important 
than others Some societies expect educational institutions to achieve a whole range of 
goals having to do with physical and social as well as educational development, while 
others confine their focus to a narrower set of learning tasks. Within each society, 
educators establish specific objectives for what is taught and define a sequence for 
instruction; these factors differ from place to place. This variation is understandable and 
reasonable and underscores the folly of viewing comparative achievement results as an 
olympiad with narrowly defined rules and criteria. 

In truth, the only justification for the disruption of student and professional lives 
caused by an international assessment is the improvement of learning. Results should 
provide teachers, school administrators, policymakers, and taxpayers with information 
that helps to define the characteristics of successful student performance and suggests 
areas for possible improvement and change. Evidence for success can be found not only 
in achievement results but also in student attitudes and perceptions, in teachers' 
instructional practices, in curricula** emphases, and in societal values as expressed by a 
cultures support for education. These "educational indicators" are increasingly the focus 
of international research in education. 1 

Nonetheless, the competitive instincts of most modem societies typically cause them 
to view survey results as challenges to better performance. While most countries reason- 
ably insist on fair and complete descriptions of elements that can explain differential 
performance, there remains the ambition to do better To the extent that these ambitions 
are tempered by realistic understandings of important differences in societal goals, 
demographic characteristics, educational systems, and economic resources, they can be 
positive forces for building bridges to improved educational performance. 

The International Association for the Evaluation of Educational Achievement (IEA) 
has demonstrated the value of comparative educational data in several previous studies. 2 
IEA studies often involve 15 to 25 different countries, some with considerable expertise 
in measurement and statistics and others without They start with an extensive process 
for defining the content-area domain, develop new test questions, and often span eight 
to ten years from start to finish. 



Organization for Economic Coctrcration and Development, "International Educational Indicators A Working Plan." APril, 

3 Recent IEA mathematics and science studies arc reported in Internationa! Association ibr the Evaluation of Educational 
Achievement, The Underachieving Curriculum: Assessing US. School Mathematics from art fnUnjationat Pcrsfleethw*SliPnsi 
Publishing ComPany, ChamPaiiin. Illinois. 1987 and International Association for the Evaluation of ICJucatbnal Achievement. 
Science Achievement in Seventeen Countries: A ?*relimiJ7<iry Report Ptirgamnn Pre*s. Oxford. England 
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The subject of this report, the International Assessment of Educational Progress 
(IAEP), involved five countries and four Canadian provinces with extensive experience in 
large-scale assessment. The project was designed to capitalize on the content and 
technology of the United States' National Assessment of Educational Progress (NAEP). 
Through the use of existing assessment questions and procedures, significant cost and 
time savings were possible. As the pace of change accelerates and as more countries 
implement educational reform, these highly efficient procedures for monitoring interna- 
tional progress may become increasingly useful 



About the Proi^cS: 



Five countries and four Canadian provinces participated in the International Assess- 
ment of Educational Progress flAEP), and in three case*; provinces assessed two separate 
language groups. 3 Results of 12 student populations are therefore presented in this 
report: 1 



British Columbia 

Ireland 

Korea 

New Brunswick 
(English) 



New Brunswick 
(French) 

Ontario (English) 

Ontario (French) 

Quebec (English) 



Quebec (French) 
Spain 

United Kingdom 4 
United States 



From each population, a representative sample of 13-year-olds was assessed in mathe- 
matics and science. Samples were drawn at random from about 100 different schools 
selected with probability proportional to their size and included about 2.000 students. In 
the United States* the sample size was about L0O0 students from 200 schools. A total of 
approximately 24000 students was surveyed. School participation rates ranged from 70 
to 100 percent, and student participation rates, from 73 to 98 percent* 

Students were administered a 45-minute mathematics assessment consisting of 63 
questions and a 45-minute science assessment made up of 60 questions. Items were 
selected from the total pool of 281 mathematics and 188 science questions used in the 
1986 National Assessment of Educational Progress (NAEP,\ Questions were translated 
from English to French, Korean, and Spanish and then independently translated from the 
non-English language back to English. The back -translated versions were compared with 
the original English to ensure that the translations were accurate. Questions were also 
adapted for cultural differences. For example, units of measurement, the names of 
children, and species of plants and animals were changed to reflect local usage and 
environments. Students also answered questions about their school experiences and 
attitudes toward mathematics and science, and their teachers rated students exposure to 
the concepts tested by the items. 



'Canada does nol have a federal system of education, 
'The United Klnftfom samPk wis drawn front htudtnta in l£n£kmd. Scotland, and 
^ec Procedural Appendix, p. 84-85. 
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The cliche has it that "comparisons are odious," They are also very difficult to make fairly 
and accurately, especially when human behavior is involved. Nonetheless, if rigorous 
procedures are followed and if the [imitations of comparisons are kept in mind, compar- 
ative studies can provide invaluable information. The consistently high interest in the 
results of the 1EA studies has demonstrated this repeatedly. 

A host of factors must be considered when comparing the achievement levels of 
students from different provinces or countries. This project was ab!e to collect informa- 
tion on only a few of these important variables: 

□ At what age do children begin school and how long is the school year? 

□ What concepts and skills have been taught by age 13? 

□ What practices do teachers use in the classroom? 

□ What home experiences support learning? 

□ What are students attitudes about mathematics and science? 

□ What is the value placed on education in each of the various societies? 

Varying achievement levels in a subject or in one aspect of a subject can legitimately 
be explained by any one of these factors or a combination of them. Survey data cannot 
establish causal relationships, but they can provide a context for examining achievement 
results and suggest Questions for further study by policymakers* educators, and 
researchers. 

Even with uniform procedures and careful monitoring, the goal of complete compara- 
bility is difficult to attain. Often local conditions necessitate modifications or compro- 
mise. This project insisted on uniform sampling procedures, high participation rates, 
standardized administration procedures, and rigorous data-analysis protocols. Specifics 
of the IAEP project implementation and data analysis are presented in the Procedural 
Appendix and a more detailed explanation is provided in the separate IAEP Technical 
Report. 



HIGHLIGHTS 



E3 in Korea, 78 percent of the 13-year-olds can use intermediate mathematics skills to 
solve two*step problems (Level 500 on a hypothetical 1,000 point scale) compared 
to only 40 percent of their counterparts in Ontario (French) and the United States. 

E3 Forty percent of Korea s 13-year-old students understand measurement and geome- 
try concepts and are successful at solving even more complex problems (Level G00K 
Less than 10 percent of those from Ontario (French land the United States have the 
same level skills. 



□ Korea's 13-year-olds demonstrate the highest overall mathematics achievement 
well above the mean of all participating countries and provinces. The other 11 
populations cluster themselves into three lower-performing groups, 

ED Four countries and provinces perform above the mean: Quebec (French), British 
Columbia, Quebec (English K and New Brunswick (English). Achievement is about 
at the mean for five populations: Ontario (English), New Brunswick (French h Spain, 
the United Kingdom, and Ireland. Finally, below the mean are students from Ontario 
(French) and the United States, 

S3 Thirteen-year-old boys and girls perform about at the same level in 10 of the 12 
populations assessed. The two exceptions are Korea and Spain, in which boys 
outperform girls, 

E3 Classroom instructional practices are similar in all of the countries and provinces 
assessed with most 1 3-year-olds ( more than 70%) reporting that they regularly listen 
to teacher lectures and work mathematics problems on their own. Fewer students 
(less than one-third) report working in small groups on a regular basis, 

03 Despite their poor overall performance, about two-thirds of the United States 
13-year-olds feel that "they are good at mathematics" Only 23 percent of their 
Korean counterparts, the best achievers, share the same attitude- 

□ In about half of the comparisons, students perform better in various mathematics 
topics than their teachers' opportunity-to-learn ratings would suggest. 

Science 

□ More than 70 percent of the 13-year-old3 in British Columbia and Korea can use 
scientific procedures and analyze scientific data (T^evel 500 on a hypothetical L000 
point scale), while only about 35 to 40 percent of their peers in the United States, 
Ireland, Ontario (French), and New Brunswick (French) demonstrate the same 
degree of competence. 
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□ More than 30 percent of British Columbia's and Koreas students are able to apply 
more advanced scientific knowledge and principles 4 r .evef 500). compared to less 
than 10 percent of their counterparts in Ireland, Ontario (French), and New 
Brunswick (French), 

□ British Columbia's and Korea's 13-yeapolds perform well above the mean on the 
science assessment; the other ten populations divide themselves into two distinct 
groups in terms of achievement 

□ Half the populations perform about at the mean: the United Kingdom, Quebec 
(English), Ontario (English), Quebec (French). New Brunswick (English), and 
Spain. A second group of four, the United States, Ireland, Ontario (French), andiNew 
Brunswick (French), perform well below the mean, 

□ Thirteen-year-old boYs outperform their female counterparts in science in ail 12 
populations assessed except the United Kingdom and the United States. The 
greatest difference was in Korea, where boys outperform girls by nearly 40 scale 
points. 

□ Students in the United Kingdom report the most involvement with hands-on 
science experiments and those in New Brunswick (English) and the United States 
report the least involvement. 

□ In all populations except Spain, students generally perform better in science topics 
than their teachers 1 opportunity-to-learn ratings would suggest. 

General 



□ In all countries and provinces assessed, the greater the amount of time spent 
watching television, the poorer student performance is in both mathematics and 
science. Survey data do not address cause and effect, 

□ Except in Spain and Ireland, more than 50 percent of the 13-year-olds report 
spending one hour or less each day doing homework assignments for all of their 
school subjects combined. The norm for Spanish and Irish students is two or more 
hours per day. 
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PART 1 

MATHEMATICS 



Mathematics: The Findings 



Average Mathematics Proficiency 



vera]] performance on the mathematics questions is summarized as an avera^j 
proficiency score for each of the 12 populations assessed (FIGURE 1.1). This score is 
expressed on a hypothetical scale that ranges VomO to L000, with a mean of 500 and a 
standard deviation of 100* 

Koreas 13~year-olds achieved the highest average mathematics proficiency score, 568. 
well above the mean of SCO, The other 1 1 student populations clustered themselves into 
three lower-performing groups. 7 

Quebec (French), British Columbia, Quebec (English), and New Brunswick (English) 
all performed above the overa!l average. Ontario (English), New Brunswick (French), 
Spain, the United Kingdom, and Ireland performed about at the mean forali populations. 
The lowest achievement was found in Ontario (French) and the United States, both with 
averages below the mean. 



*The mathematics proficiency scale developed u£in£ item respon&e theory. One ot the mathematics questions was 
excluded from the scale because its pattern of perform a nec - ariedton^iderahlyiicro&s population^, The reference tfroup for the 
mean and Standard deviation is the estimated total number of 13-year-olds across all 12 popul^littrtstithctul 5384.UO0 students). 
More than percent of the students' scores fall within .he ranffc of 200 1o 800. See the Proved oral \ppendix and the IARP 
Technical Report for a discussion of s^lc con struct! or. and differential item functioning. 

'Com pari son & of levels of proficiency amonfi the 12 populations were conducted usint! a generalized Tjmhjnes multipk- 
comparison procedure lor mt h an& with unequal variances Ajit C. Tamlune. "A Comparison of Procedure* iff Means with 
Unequal Variances", Journal of the American Statistical Association, ]970 h 71 pp 471-480. 
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500 
400 
300 



mm 

§8 



-u, ' 



If 



: -?J--l, J ^.i.>j- 



KOREA 

567.8 
(27) 



i r 

NEW 
BFUNSMCH 
(ENGLISH) 

543.0 5398 535.8 529.0 
(3.1) (2.2} (20} (2.6) 



QUEBEC BRUSH QUEBEC 
(FftEttCHl COLUMBIA (EMGUSHj 



(FRENCH) 

516.1 514.2 



SPAIN 


UNITED 

kingdom 


IRELAND 


ONTARIO 
(FRENCH) 


STATES 


511.7 
(4.6) 


509.9 
(3.5} 


504.3 
(3.7) 


481 5 

(2.7) 


473.9 
<4.5) 



* Differences in performance between the four groups are statistically significant at the .05 level; differences in performance within groups 
are not statistical/ significant. Jackknifed star.dard errors are presented in parentheses. 



Differences among the four groups are statistically significant that is, there is a very low 
probability (less than 5 percent) that the observed differences are caused by uncertainties 
associated with sampling. These differences can be taken as reat. However, within each 
of these groups, the differences in performance are not great enough to be significant. 
The average proficiencies for populations within each of the four groups are essentially 
equal 

Every statistic computed for this report carries its own error of estimation or standard 
error, usually expressed as plus or minus a specified number. This indicates that there is 
a 68 percent chance that the true value is within the range of the number plus or minus 
one standard error, and a 95 percent chance that it is within the number plus or minus 
two standard errors. In this report, the standard error is either represented graphically, 
written within parentheses next to or below each statistic, or the range is specified. 
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The HAEP Mathematics Proficiency Scale 

When examining achievement results the leader is always left with the question, "Is 
this good enough or should more be expected?'* In an attempt to make the mathematics 
assessment results more meaningful, the project has defined five points along the 
mathematics proficiency scale — 300, 400, 500 T 600 T 700 — in terms of what students 
know or can do if they perform at that level. These "anchor*' points or levels are also 
illustrated by sample questions chosen from the assessment to represent the kinds of 
tasks students at each level typically can address successfully. 

With anchored proficiency scales, it is possible, for example, to inform the public that 
85 percent of the 13-year-olds in a country or province can successfully carry out basic 
arithmetic operations, while only 40 percent can solve two-step problems. With this kind 
of information, educators and policymakers can make judgments about the adequacy of 
the mathematics skills of their young people, who face the demands of an increasingly 
technological and competitive world. Governments concerned about quality-of-Hfe issues 
or the development of a nation's human resources can also interpret such findings in 
light of social or economic plans. 

Anchor points along;* performance scale can also be used as targets or goals for schools 
as they o!an their programs for the future. Policymakers can consider* for example: 

□ What percentage of their i3-year-olds should be able to use basic operations 
and solve simple problems (i.e-, achieve at Level 400)? 

□ What p?rcentage should be able to understand measurement and geometry 
concepts and solve more complex problems (i.e.. master the skillsat Level 600)? 

FIGURE 12 describes five levels of the IAEP mathematics scale and presents sample 
questions taken from the assessment. 

o J 8 
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LEVEL Perform Simple Addition and Subtraction 

300 Students at this level can add and subtract two-digit num* 
5ers without regrouping and solve simple number sen- 


^ * □ * 16 

Wh<H numk'i shnuld Co m th^ fro* to moke 
ii'nU'jK£ jhivt' TRUE' 

ANSWER _ 


(he number 


tences involving these operations. 






LEVEL Use Basic Operations 


T 




to Solve Simple Problems 

Students at this level can select appropriate basic opera- 
tions (addition, subtraction multiplication, and division) needed to 
solve simple one-step problems. They are capable of evaluating simple 
expressions by substitution and solving number sentences. They can 
tocate numbers on a number line and understand the most basic 
concepts of logic- percent- estimation, and geometry. 


0 1: 

Whit numhui dues T point tn- 
G> ] 

0 3 
©J 





LEVEL Use Intermediate Level Mathematics Skills 
i-aa 10 Solve Two-Step Problems 

500 

Students at t k is level show growth in aif mathematics topics 

in ihe assessment, They demonstrate an understanding of the concept 
of order Place vafue, and the meaning Of remainder in division, they 
know some properiies of odd and even numbers and of zero; and they 
can apply elementary concepts ot ratio and proportion. They can use 
negative and decimal numbers; make simple conversions involving 
fractions, decimals- and percents: and can compute averages. Siudents 
can use these skiiis to solve probiems requiring two or more sieps and 
can represent unknown Quantities with expressions involving vari- 
ables. Students can measure length, appfy scales, identify geometric 
figures, calculate areas of rectangles, and are able to use information 
obtained from charts, graphs, and tables. 



LEVEL 

600 



Understand Measurement and Geometry 
Concepts and Solve More Complex Problems 

Students at this level know ho v to multiply fractions and 
decimals and are able to use a range of procedures to solve more 
complex problems. Students demonstrate an increased understanding 
of measurement and 9eometry concepts. Tney can measure angles 
found in simple figures, understand various characteristics of circles 
and triangles, can find perimeters and areas, and calculate and com- 
pare volumes of rectangular solids. Students are also able to reco9nize 
and ex l end number patterns. 



LEVEL 

700 



Understand and Apply More 
Advanced Mathematical Concepts 

Students at ttiis level have me ability lodeal wilh Properties 
of the arithmetic mean and can use data from acomPtex table tosolve 
problems. They demonstrate an increasing ability to apply school- 
based skills to out-of- school situations and problems. 



Hac an: ihc of five children. 

13. it. t. 4 r 4 
What is the avtrdPi. jftc of ihoe children? 
G> 4 
<3 ft 
7 

e> » 
0 9 

© 1 don't knuw 




The knfiih ot t side of this. Wujro 6 What is the mJiui* ot the crrcle? 
OZ CD 4 Oft <I> » 09 <3> J don t know. 

NUTtillrVE VALUE QF CERTAIN FOODS 







C4IOr,?l 


PlQT^Ifl 


CarootiyC'die; 


Havana, fin 


1 


100 


1 


2G 


B**l himtju r qr, 


3 Of 


?45 


21 


0 


WHolf rrtilli 


1 CufP 


150 




12 




1 


1?S 


T 


IS 


£99*. t»il«J 




ISO 


13 


1 



According 10 ihf uhle, what is thtr ioijI nimuunt t»t prott'm cnnuincd in 
iwo boik'd cK£s and onu-half cuP of whoti: mllk : 

ANSWER - 
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Tfce Power of the Scale 



Overall averages or means often mask important distinctions. Examining the percent- 
ages of the student populations achieving at or above each of the five descriptive scale 
points permits a consideration of more useful information. Many 13-year-olds are within 
one or two years of completing their study of mathematics. While it can be comforting to 
learn that almost 100 percent of a country s 13-year-olds have mastered basic addition 
and subtraction skills, it may be of concern that only 40 percent can use fractions, 
decimals, and percents. since the 60 percent of students who have not yet developed these 
skills may experience difficulty with secondary-school mathematics and, if their compe- 
tence is not increased, may face serious problems dealing with the everyday quantitative 
problems that confront modern adults. 

The results displayed in TABLE 1.1 detail the differences in performance among the 
various countries and provinces at each proficiency level. 

AM of the countries involved in the assessment share common goals for an improved 
quality of life for their citizens and for successful economic achievements in the world 
arena. Each society is experiencing rapid technological change that often translates into 
the need for employees who are better trained in mathematics and science. The 13-year- 



RPercerita^es Pe 


irforming 


At o r Ah on 


r e . - ;v: 








IS Math en 


lati'cs Sea 


le, Age i: 


5 : .;. 






Add 
and 
Subtract 


Simple 
Problems 


Two-Step 
Problems 


Understand 
Concepts 


Interpret 
Data 


LEVEL & 


300 


400 


500 


G0Q 


700 


Korea 


100 


95 


78 


40 


5 


Quebec (French) 


100 


97 


73 


22 


2 


British Columbia 


100 


95 


69 


24 


2 


Quebec (English) 


100 


97 


67 


20 


1 


New Brunswick (English) 


100 


S5 


65 


18 


1 


Ontario (English) 


99 


92 


58 


16 


1 


Mew Brunswick (French) 


100 


95 


58 


12 


<1 


Spain 


99 


91 


57 


14 


1 


United Kingdom 


98 


87 


55 


18 


2 


Ireland 


98 


86 


55 


14 


<1 


Ontario (French) 


99 


85 


40 


7 


0 


United States 


97 


78 


40 


9 


1 


Jackknifed si an cf a rd errors for percentages range 1 rom 1 ess than ,1 to 2 4 and are Provided m me Gala APPendi*. 
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olds of 1988 will be the 18-year-old workers of 1993 — a not very distant future. If more 
than 75 percent of the 13-year-olds in a country are competent in intermediate mathe- 
matics skills, does that country have a significant social or economic advantage over a 
country in which only 40 percent of this age group has attained this proficiency level? 

From another perspective, school administrators may need to question whether poor 
performance of 13-year-olds reflects a generally inadequate program or simply a situation 
in which concepts have not yet been taught because of the sequence of the curriculum. 
Performance results may signal the need to reconsider specific syllabi or broad educa- 
tional standards. 

The mathematics results illustrated in TABLE LI demonstrate major differences in the 
percentages of 13-year-olds in various countries and provinces who have mastered 
different skill levels. Ninety-five percent or more of the students in Korea, Quebec 
(French), British Columbia, Quebec (English), New Brunswick (English), and New 
Brunswick (French) can use basic operations to solve simple problems (Level 400) 
compared to only 78 percent of their peers in the United States, Between 35 and 92 
percent of the students from the other five populations demonstrate these same Level 
400 skills. 

Seventy-eight percent of Korean 13-year-olds can use intermediate skills to solve 
two-step problems (Level 500). In contrast, only 40 percent of their age-mates in Ontario 
(French) and the United States are able to achieve at that level or higher. The percentages 
of students in the other populations at Level 500 or above range from 55 to 73. 

An impressive four out of 10 of Koreas 13-year-olds understand measurement and 
geometry concepts and can apply a range of problem-solving strategies to more complex 
problems (Level 6001 Two out of 10 of British Columbia's and Quebec's (French and 
English) students did as well compared to fewer than one out of 10 of their peers from 
Ontario (French) and the United States. 

The variation in performance among the countries and provinces is considerable at 
Level 500, reflecting a spread of 38 points between the highest and lowest percentages 
of students achieving at this level or above. At Level 400, there is a 19 point difference, 
and at Level 600, a 33 point discrepancy. These results suggest that in many countries 
there is a great deal of room for improvement in the way students are prepared for 
secondary school, especially with respect to those intermediate mathematics skills that 
are usually the focus of study in the middle-school years. 

The Gender Gap 

Many educational research studies have found performance differences between teenage 
boys and girls in mathematics. 8 The findings of this International Assessment of Educa- 
tional Progress suggest a different picture, with 13-year-old boys and girls performing 
about at the same level in 10 of the 12 populations assessed ( FIGURE 1.3). Only in Korea 
and Spain do boys at this age achieve significantly higher in mathematics than do girls. 



H John A, Dossey. Itia V.S, Mulli&Maty M. Undquist. and Donald L Chambers. The Mathematics Report Cord: Arc lie Measuring 
UP? Trends and Achievement Based on the 1986 Nathmt Assessment ^atiunal Assessment of Educational Progress. 
Educational Testing Service. I93S 
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600 



500 



40O 



5* 

J? 



MALES 
FEMALES 



Statistically signsficant difference 
between groups at the .05 Jevel. 




MALE 



FEMALE 



KOREA 


QUEBEC 


BRITISH 

coubbw 


QUEBEC 


576.7 


5463 


539.6 


534.1 


0.4) 


(4.2) 


{2.8) 


(3.0) 


558.0 


539.3 


541.3 


537.3 


(3,9) 


C3.0) 


(2.6) 


(2?) 



NEW 
BRUNSWICK 
(EHGUSHjt 

526.6 

(4.31 

529.0 

(3.4) 



ONTARIO 

517.8 
(4.4) 

514.6 
(3.3) 



BfUfflSWO 
(FRENCH) 

516.8 

(4 2) 

513.4 

(3.6) 



SPAN 

523.2 
/5.3I 
199 9 
(5.0) 



UNITED 

XWGDOM 

507.0 
(5.0) 

512.5 
(3.9) 



IRELAND 

508.2 
(5.7) 

499.6 
(3.4) 



ONTARIO 
(FRENCH) 

460.6 
(3.4) 

462.7 
(3.0) 



UNHID 
STATES 

474.6 
(6.4) 
4732 
(5.1) 



* Jackknifed standard errors are Presented in Parentheses, 

* Background data are missing from 31% ol the New Brunswick {English) students. The resultant stfect on the differences between groups tor 
this particular province is estimated to be less than one standard error 



Where Bo We gfcand? Are she Results Good Enough? 



Clearly both Questions are appropriate and equally Important in every country and 
province. The first one may have greater political significance, but answers to the second 
address the Qualitative issues that permit policymakers and citizens to make informed 
choices about priorities that will affect their futures. 

Thirteen-year-olds who have mastered the skills reflected in the descriptions of Levels 
600 and 700 probably represent the pool from which most of tomorrow s mathematicians, 
engineers, and scientists will emerge. Do these results predict that certain populations 
will be responsible for amajority of the important achievementsin these fieldsduring the 
21st century? Obviously, the answer to this question depends on the opportunities 
presented in each society and the support available to young people in each country to 
pursue and develop their interests. 
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MATHEMATICS 



CHAPTER TWO 

Mathematics: Instruction 
and Attitudes 



— he information gathered from students about their attitudes toward mathematics, 
how they spend their time in class, and the amount of homework they do after school is 
instructive. It is also confusing. Groups of students who do very well share attitudes and 
learning experiences with other populations who perform much less well on the assess- 
ment For example, most students (more than 70%), whether they score high or low, 
think mathematics is useful in solving everyday problems, and listen to their teacher 
explain a mathematics lesson several times a week or more. Students from the best- 
performing population report doing more mathematics homework than is typical as do 
students from one of the average- performing populations. 9 

Learning Matfeematscs 

Classroom Activities, This study asked students a series of questions concerning 
their mathematics classroom activities* and TABLE 2.1 presents some of the highlights. 
Entries reflect percentages of students reporting frequent classroom activity, "almost 
every day" or "several times a week/' of the types described. Other options provided to 
students were "once a week " "less than once a week/ 1 and "never." The frequency of 
these activities is not consistently related (positively or negatively) to achievement 
among these groups, so the performance data are not presented. For example, in some 
populations high performance is associated with listening to the teacher explain a 
mathematics lesson several timesa week or more. In other populations high performance 
is associated with listening to the teacher only once a week, and in still others the high 
performers are those who report listening to the teacher less than once a week. 



y Jn reading this chapter, note that background data are missing from 31% of the New Brunswick (English) students. The 
resultant effect on the differences among groups for this particular province is estimated to be less than one standard error. 

20 21 
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! 
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Listen lo 




Work 


Gel individual 


Get help 






tn eleacher 




mathematics 


help Irom 


in 


Help a 




explain a 


mathematics 


problems 


teacher 


mathematics 


classmate 




m athp mat (re 




in 




Jj om 3 


do 




lesson 


alone 


smaU pro u Ps 


mathematics 


classmate 


mathematics 


Korea 


71 (1.4) 


76 (1.1) 


15 (1.4) 


6 (0.6) 


38 (1.3) 


33 (1.3) 


Quebec (French) 


97 (0.4) 


89 (0.8) 


16 (10) 


24 (1.1) 


17 (0.9) 


21 (0.8) 


British Columbia 


94 (0.6) 


90 (0.7) 


15 (10) 


24 (0.9) 


35 (1.1} 


46 (1.1) 


Quebec (English) 


94 (0.7) 


90 (0.7) 


13 (0.8) 


23 (1.1) 


27 (0.9) 


46 (1.2) 


New Brunswick (English) 


93 (0.7) 


91 (0.7) 


11 (10) 


21 (1.5) 


28 (1.3) 


39 (1.4) 


Ontario (English) 


95 (0.6) 


86 (1.0) 


17 (1.2) 


20 (1.2) 


32 (1.4} 


46 (1.5) 


New Brunsurck (French) 


97 (0.5) 


93 (0.6) 


17 (1.8) 


28 (1.6) 


22 (1.8) 


28 (1.7) 


Spain 


9a (0.4) 


93 (0.8) 


31 (17) 


28 (1.5) 


33 (1.3) 


53 (1.7) 


United Kingdom 


76 (1.9) 


78 (0.8) 


14 (0,9) 


31 (1.5) 


31 (1.2} 


40 (1.2) 


Ireland 


96 (0.5) 


80 (0.9) 


17 (1,3) 


19 (1.1) 


15 (0.8) 


27 (1.4) 


Ontario (French) 


96 (0.5) 


84 (1.1) 


22 (1.3) 


19 (1.0) 


30 (1.4) 


34 (1.4) 


Uniled States 


93 (1.0) 


92 (0.9) 


20 (1.7) 


29 (2.1) 


24 (1.6} 


35 (1.4) 


* Jackkmfed standard errors are Presented in Parenlheses. 
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The results indicate variety in instructional patterns across the populations. However, 
some classroom activities emerge as typical for all participants, particularly lecturing and 
seatwork. Most 13-year-olds (more than 70%) in all countries and provinces report that 
they regularly listen to teacher lectures and work mathematics problems on their own. 
Except in Spain, less than one-quarter of the students report working in small groups on 
a regular basts, a technique thought to improve performance and strongly recommended 
by many mathematics educators. About a third of the Spanish students say they work in 
small groups at least several times a week. 

A very small percentage of Korean students (6%) report getting individual help from 
their teacher several times a week or more, probably a reflection of class sizes of 40 to 55 
students. Indeed, less than a third of the students in any country or province say they 
regularly get individual help from their teachers. 

In 8 of the 12 populations, between 25 and 40 percent of the students report that they 
interact with their peers at least several times a week, seeking assistance with their 
mathematics assignments. In the remaining four groups, less than one-fourth do so 
regularly. Offering to help a classmate with mathematics is reported as a common 
practice by about 40 to 55 percent of the students in six countries or provinces, while in 
the six other groups, between 20 and 35 percent report doing so regulariy. 
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Homework, In addition to classroom activities, students all over the world reinforce 
their learning by doing homework assignments. Students in participating countries and 
provinces differ in the amount of homework they do (TABLE 2.2), Thirteen-year-olds 
from most groups do not spend much time on homework. Except in Korea and Spain, 
between 40 and 65 percent of the students report doing less than one hour of mathemat- 
ics homework each week. Time spent on mathematics homework ts only slightly higher 
for Korean and Spanish children. Forty-five percent of the Korean studei ts and 37 
percent of the Spanish students say they do one to two hours of homework in this subject 
weekly. 



{ Percentages Reporting Amounts-of ■/]■■ ~1§ 

•^Weekly Mathematics Homework, Age 13* / s g|§ 



S) 





Less than 1 hour 


1 to 2 hours 


3 or more hours 


Korea 


32 (1.3) 


45 (13) 


23 (14) 


Quebec (French) 


44 (17) 


36 (1.2) 


20 (13) 


British Columbia 


44 (1,4) 


36 (10) 


21 (11) 


Quebec (English) 


43 (1.2) 


35 (11) 


23 (1>1> 


New Brunswick (English) 


49 (1,6) 


31 (13) 


20 (0,9) 


Ontario (English) 


43 (1-3) 


37 (11) 


20 (1,1) 


New Brunswick (French) 


44 (1,3) 


30 (14, 


26 (16) 


Spain 


35 (1,8) 


37 (12) 


23 (16) 


United Kingdom 


56 (15) 


38 (12) 


6 (0.6) 


Ireland 


64 (14) 


23 (10) 


14 (10) 


Ontario (French) 


55 (14) 


29 (11) 


16 (14) 


United States 


62 (18) 


28 (15) 


11 (14) 


•Jackknifed standard errors are Presented in Parentheses. Percentages do not always sum to 100 due to rounding. 



The results in FIGURE 2.1 show that for some countries and provinces, including 
Korea and Spain> inct'eaied time spent on mathematics homework is positively associated 
with higher mathematics achievement and for some groups it is not This inconsistency 
is not surprising, considering the many purposes for which homework ts used. In some 
cases equal amounts of homework are assigned to all students; in others homework is 
emphasized in advanced classes. In still others, extra homework is assigned to lower- 
achieving students in an effort to improve their performance. 



9 

ERLC 



22 



ft KJ 



Average Proficiency by Reported Amounts 
of Wee % .Math ematics Homework, Age 13' 




QUEBEC 



COLUMBIA 



Quebec 



QfflMO 



uwited 



1 

I * JachknHed standard errors for proficiencies range from 3.0 to 10.8 and are provided tn the Data Appendix. 
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Altitudes Toward KfaUiemaises 



Students were also asked about their attitudes toward mathematics, and TABLE 23 
presents data on the percentage of students who say they "strongly agree" or +t agree ,+ to 
statementsabout mathematics, or respond that they'like mathematicsa lot" or + 'a little" 



^Percentages Responding . ; -pp$jti y 6 ly^fb r/-.,-'^ : --= V: i Vf; ; ,'J" - ^^ 
C Mathematics Attitude Questions, Age 13* j S 





5> 


UK! 1 













Mathematics Is 
useful in solving 
everyday 
problems. 



Mathematics is 
' more lor boys 
than 
tor gills. 



am good 
at 

mathematics. 



Hike 
mathematics 
a little or 
a tot. 



Korea 

Quebec (French) 
British Columbia 
Quebec (English) 
New Brunswick (English) 
Ontario (English) 
New Brunswick (French) 
Spain 

United Kingdom 
Ireland 

Ontario (French) 
United States 



87 (0.7} 

78 (1.0} 

76 (1.0) 

78 (0.9} 

78 (1.0} 

84 (0.9) 

79 (1.4) 

85 (1.1) 

80 (0.9) 
80 (0.8, 
82 (0.9) 
76 (1.8) 



23 (0.9) 

38 (1.1) 
3 (0.4) 
3 (0.3) 
3 (0.4) 

3 (0.5) 
35 (1.5) 

6 (0.7) 

4 (0.4} 

7 (0.7) 

39 (1.2} 
6 (1.1) 



23 

58 
57 
65 
62 
66 
59 
60 
47 
49 
63 
68 



(1.1) 
(1.2) 
(1.0) 
(1.0) 
(1.6) 
(1.4) 
(1.3) 
(1.6) 
(1.3) 
(1.2) 
(1.2) 
(1.1) 



72 (1.2) 

83 (1.0) 

64 (1.0) 

69 (1.4) 

71 (1.5) 
74 (1.1) 

81 (1.3) 
68 (1.5) 
80 (1.0) 
77 (1.0) 

82 (0.9) 

72 (1.3) 



"Jackkmled standard errors are Presented in parentheses. 



Most students (more than 75%) in all participating countries and provinces agree that 
mathematics is useful in solving everyday problems, suggesting that its importance and 
utility are widely accepted. 

More than one-third of the students in the three French-speaking populations, and to 
a lesser degree the Korean students, agree that mathematics is more for boys than for 
girls. However, the translation of this question into French may have projected a more 
abstract concept, literally "mathematics is more appropriate for boys than for girls," and 
this may account for the differential responses of the French-speaking students. More- 
over, this attitude is not reflected in differences in performance of males and females in 
these populations (see FIGURE 1.3). 

About two-thirds of United States' 13-year-olds feel they "are good at mathematics'* 
despite their poor overall performance, while only 23 percent of the Korean students, the 
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best performers, have that same attitude. Nevertheless, students in all groups who give 
a positive response to this statement are higher mathematics performers than those who 
give a negative response. About 65 to 85 percent of the students from all surveyed 
countries and provinces indicate they "like mathematics" and these students are higher 
achievers than those who indicate that they do not like mathematics. 



What goes It All Mean? 

ft would be comforting to point to two or three instructional strategies or student 
attitudes that are dearly related to success. However, the lack of consistency in this type 
of data is understandable, because of cultural and curricular differences as well as the 
difficulty of isolating factors that are clearty associated with mathematics performance. 
Seeking helpfromateacherorclassmatemaybe encouraged in certain environments and 
impossible in others. The amount of homework may reflect the diligence of a good 
student or the penalty for poor classroom performance. Cultural practices may affect the 
answer to the question, "Are you good at mathematics?" For example, Korea s researchers 
suggested it would be against their tradition of humility for many of their students to 
answer "yts" to this question. 

Nonetheless, if these data arouse teachers' curiosity about their own practices or about 
the attitudes and perceptions of their students, positive change can and probably will 
occur. Even at this modest level, these findings may provide important clues to the 
eventual solutions of some of education's persistent problems. 
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PART I j 

MATHEMATICS 




Mathematics: Topics 

HP 

Jihe sequence in which topics are presented and studied in each educational system 
is usually governed by a published or generally accepted syllabus or course of study. 
Guidelines are often established by committees of teachers and subject-matter experts, 
periodically revised as research and experience suggest improvements, and usually 
codified in textbooks and teachers' guides. These materials typically describe objectives, 
define sequences of topics to be presented, and often detail the relative emphasis to be 
applied to one content area as compared to another. These syllabi represent what is often 
referred to as the 'Intended curriculum" as different from either the "implemented" or 
the "achieved" curriculum. 

The curriculum is "implemented" at the school and classroom level. This is where 
principals and teachers identify priorities for instruction, employ specific instructional 
materials, and use their own preferred teaching strategies. These local choices may 
restrict or expand students opportunities to learn various con tent areas. A measure of the 
students' opportunity to learn is often obtained from teachers, who indicate whether or 
not their students have been taught the concepts tested by a particular question in an 
achievement test. 

The third level of the curriculum is the "attained" curriculum. This refers to the 
knowledge and skills that actually have been learned by the students. How well the 
students do on achievement tests usually serves as a measure of the attained curriculum. 

Two measures of curriculum are reported in this chapter. First, achievement results 
(the attained curriculum) are reported for six mathematics topics. Second, teachers' 
ratings of students' opportunity to learn the concepts tested by the items (the imple- 
mented curriculum) are presented for the same content areas. Information on the 
intended curriculum is presented in Part III in separate descriptions of the cultural and 
educational contexts of each participating country and province. 



The 62 mathematics questions included in the final analyses of the IAEF were divided 
into six topics, as indicated in TABLE 3.1. 10 In the results that follow, achievement within 
each of these areas is described as the average percent of correct answers for the 
questions that measure the topic 11 The proficiency scale used in the previous chapter 
summarizes overall performance in mathematics. The average percents correct describe 
achievement in content areas that are often included in the mathematics curriculum. 
Within each topic there are items that reflect many different levels of proficiency from 
low to high. 

Unlike the scale scores reported earlier, average percents correct are influenced by the 
difficulty of the particular questions. Because of the variance in question difficulty, it is 
not appropriate to compare the level of the average percents correct in one topic, for 
example, Numbers and Operations, with those in another topic, for example, Geometry. 
Comparisons are indeed meaningful between the average percent correct of one country 
or province and that of another within the same content area. Because the number of 
questions within each category is relatively small, the results obviously do not represent 
a comprehensive assessment of that topic. 
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24 


6 


3 


10 


6 


8 


62 



*[n the analysis, one Question was droPPed irom the 63-item mathematics assessmenl tecause ol dif fereniial iiem lunctiomng, to leave a lotal 
of 62. 




10 ln the analysis, one question was dropped from the 63-it em mathematics assessment because of differential item functioning, 
to leav'e a total of 62, 

n The number of questions within each topic was insufficient for developing toPic-relatcd proficiency scales. 
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dumber 5 and Operations 

This topic covers concepts of whole nunv 
bers, common fractions* decimal fractions, 
integers, and percents. Items measure addi- 
tion, subtraction, multiplication, and divi- 
sion with these kinds of numbers as well as 
estimating the results of these types of com- 
putations. Properties of the number system 
and relationships such as place value, odd 
and even, the properties of zero, and ratio 
and proportion are also assessed by these 
questions. 

The average percents correct for Num bers 
and Operations are presented in FIGURE 3.L 
In general the relative performance of coun- 
tries and provinces on this topic mirrors 
their overall achievement in mathematics. 
Exceptions are apparent in the United King- 
dom, where students perform less well in 
this topic than they do overall, and New 
Brunswick (French), where students per- 
form at higher levels than they do overall, 12 
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Relations, Fanci;o2to ? 
and Algebraic SKgi'ecoic-ns 

The tasks in this category assess the use of 
variables in expressions of relationships, 
translations from words to symbols, and use 
of variables to represent properties of opera* 
tions and equality. Tasks involving solving 
equations and generalizing patterns are also 
included. 

As seen in FIGURE 3.2, Relations, Func- 
tions, and Algebraic Expressions is a topic in 
which students from the United Kingdom do 
relatively well compared with their overall 
achievement. Also, performance is higher in 
Ontario (French) than their overall achieve- 
ment and equals that of most of the popula- 
tions in this study. British Columbia's stu- 
dents achieve less well in this topic than 
they do overall 



Questions in this category measure prop- 
erties of and relationships among geometric 
figures such as circles, squares, rectangles 
parallelograms, triangles, and angles. The 
assessment results for Geometry are pre- 
sented in FIGURE 3.3. Korea, Spain, and the 
United Kingdom all perform relatively well 
in this topic compared with their perfor- 
mance overall. British Columbia and New 
Brunswick (French) students achieve at 
lower levels in Geometry than they do over* 
all. 
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Measurement 

Concepts of measurement and applica- 
tions of measurement of length, area, and 
volume are included in this topic as well as 
understanding and using scales. The average 
percents correct for Measurement items, 
FIGURE 3.4, generally mirror overall 
achievement levels. Exceptions are New 
Brunswick (English), where students per- 
form at higher levels than they do in mathe- 
matics overall and the United States, where 
students perform less well than they do 
overall 



Bafca Organization 
and Interpretation 

Questions in this category assess organiz- 
ing, analyzing, and interpreting data in 
tables, charts, and graphs. They also cover 
the concept of average. Students from 
Korea, the United Kingdom, and the United 
States perforin at relatively higher levels in 
this topic than they do generally in mathe- 
matics. New Brunswick (French) and Irish 
students perform less well on these items 
than they do overall 
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■ FIGURE 3:6 

Average ,. 
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Correct, Age 13 
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logic and Problem Solving 

Questions in this topic assess an under- 
standing of the tools of mathematics itself, 
those processes that are central to the exten- 
sion and development of mathematics and 
its use. These methods cut across all mathe- 
matics content areas. Included are concepts 
of logic, sufficiency of data, and problem- 
solving strategies. 

The results for this topic are presented in 
FIGURE 3,6, This is the one area in which 
Korean students are not the top performers, 
Quebec (French), New Brunswick (French), 
and Ontario (French) students also perform 
below their usual levels in this topic One of 
the highest achieving groups is the sample 
of students from the United Kingdom, who 
score relatively well in this topic compared 
with their performance overall Irish stu- 
dents also achieve at higher levels in Logic 
and Problem Solvtngthan theydo in mathe- 
matics in general 



Curricular differences 

The relative emphases on mathematics topics and the order in which they are 
introduced varies considerably from one country or province to another. It is likely that 
some of these curricular differences are reflected in these results by topic. Without 
further information, it is difficult to know whether relative strengths can be attributed 
to effective instruction* curricular emphases* or the sequence in which topics are 
introduced in the school curriculum. 



Opportunity to Leasna Mathematics by Topics 

Additional information regardingdifferential performance is provided by a measure of 
students* exposure to the content tested by the IAEP mathematics items. In each 
participating school, a mathematics teacher or coordinator was asked to indicate the 
percentage of the seventh- and eighth-grade students in the school that had already had 
an opportunity to learn — at any time in the school programs — the concepts tested by 
each item in the mathematics assessment 13 Response choices included "most (more than 
75 percent)," "some (25 to 75 percent)," "few (fewer than 25 percent)," and "none/' 

13 In some cases stings were thecons^nsusofseveralteachersateachSlrade.iindinsome cases more than one teacher in a Grade 
in a school Provided ratings and responses were weighted appropriately. 
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"FIGURE 3.7 



Average Perce nts Correct and 
vOpportunity-to-Learn Ratings for 
j|Six Mathematics Topics, Age 13 ; 
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r~j Average percent correct. 



Average rating for opportunity to leam and 95% confidence in- 
terval. It can be said witti 95% certainty that the average rating 
of the population is within this interval. 



NOTE: 

tf bar terminates above the dot and confidence interval 
it indicates that student performance is higher than 
teacher estimates of student exposure to tie concepts 
tested by the items 
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FIGURE 3.7 combines the achievement information on mathematics topics presented 
earlier with the average opportunity-to-learn ratings. The average percents correct i the 
bars) indicate the percentage of items in a topic that students on average have answered 
correctly. The average opportunity-to-learn rating (the dots) indicate the percentage of 
items in a topic to which students on average have been exposed. When the dot and its 
confidence interval are within the bar, student performance is higher than teacher 
estimates of student exposure to the concepts tested by the items. 

In about half of the comparisons between average percents correct and opportunity-to- 
learn ratings, students perform better in mathematics topics than their teachers ratings 
would suggest. This is true most often in Geometry and Logic and Problem Solving. 
Spain tends to have the highest estimates of opportunity to learn the material; however, 
Spanish students only score at about the overall average. 

The analyses reported here focused exclusively on the results from schools in which 
more than 75 percent of the students had already had an opportunity to learn the content 
assessed. It was assumed that students in these schools had been "exposed" to the 
content. Ratings for grade 7 and grade 8 were weighted in proportion to the number of 
13-year-olds in each of these grades. This provided an estimate of the percentage of 
questions to which an average 13-year-old had been exposed for each content area. 14 
Specifics about this measure and its analysis are presented in the Procedural Appendix 
and a more detailed explanation is provided in the separate IAEP Technical Report. 



More Evidence of Internationa! Differences 

The student performance and opport unity- to-Iearn ratings by topic highlight the 
cunricular differences between countries and provinces. Variation exists in what is taught 
and when it is taught. If decimal fractions are not taught until late in eighth grade, it is 
reasonable to expect poor performance on these questions by seventh and eighth graders 
assessed in February, 

In many cases, the results of lAEFs opportunity-to-learn analyses do not follow the 
theoretical model that suggests students only learn what they are taught in school. 
However, these results are not altogether surprising. First the ratings were global 
estimates at the school level and raters may not have been fully aware of content 
coverage in all seventh-and eighth-grade classrooms or in the lower grades. Secondly* and 
perhaps more importantly, students extend their learning outside of school and can apply 
their knowledge to many new situations, including the IAEP mathematics tasks. Also, 
some aspects of mathematics, such as logical reasoning, develop as thinking matures and 
are not necessarily specific to the mathematics curriculum, 



14 The nonreSponse rate of teachers translated into missing data for irorc than 10 percent of the student Populations in New 
Brunswick fEMhsh), Nev? Brunswick {French), Ontario (English), Ontario I French). Quebec {English), and the United States, 
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CHAPTER FOUR 

Science: The Findings 



JL erformance on the science questions is summarized as an average science profi- 
ciency score for each of the populations assessed (FIGURE 41 K This score is expressed 
by a hypothetical scale that ranges from 0 to l t 000 with a mean of 500 and a standard 
deviation of 100 + 1:j 

Two populations stand out in terms of average science performance: students from 
British Columbia (551) and those from Korea (550). The remaining countries and 
provinces fall into two lower-performing groups. 16 

Significantly below these Iwu Luf per fu fitted a group of six populations hovers around 
the 500 mean. They are the United Kingdom, Quebec (English). Ontario (English), 
Quebec {French), New Brunswick {English), and Spain. Another cluster of four popula- 
tions, the United States, Ireland, Ontario (French), and New Brunswick (French) scores 
well below the mean. Proficiencies of countries and provinces within each of these 
groups are essentially equal. 



r "The science scale was developed using item response theory. Six ofihe 60 science questions were excluded from the final scale 
because patterns of performance on these items varied considerably across pop utotions> The reference group for the mean and 
Standard deviation is the estimated total number of 13-year*olds across all 12 Populations (about 5>215>001> students). More 
than 99 percent of the students* scores fall within the range of 200 to BOO. Sec the Procedural Appendix and the 1AEP 
Technical Report far a discussion of scale construction and differential item performance. 

1 Comparisons of levels of proficiency among the 12 populations were conducted using a gene r^lizcd Tamhane's multiple-com- 
parison procedure for means with unequal variances^ A)it C. Tamhane/ A Comparison of Procedure;; for Multiple Comparisons 
of Means with Unequal Variances/' Journal of The American Statistical Association. 1979. 74* pp 171-430. 



35 




500 
400 
300 



I" 

HOPEA 
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(2.9) 



1 
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- , .< .-<- 
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T 



BRITISH 
COUJWflW 

551.3 
(2,1) 



UNITED QUEBEC ONTARIO QUEBEC 
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msm mm m\m (frewcmi 'Jj^jj 

519.5 515.3 514.7 513.4 510.5 503.9 
(3.7) (2.8) (2.7) (3.3) (2J) (4,3) 



UNITED 

STATES 

478.5 
(4.8) 



IRELAND 



469.3 
P.5) 



^= BRUNSWICK 



463.3 
(22) 



468.1 
(3.9) 



Differences in performance between the three groups are statistically significant at trie .05 level; differences \n performance within groups 
are not statistically significant, Jackkniled standard errors are presented in parentheses. 



The IAEP Science Scale 



In today's societies, it seems important that all citizens have at least a basic understand- 
ing of scientific concepts. As taxpayers and voters, they are being called on to support the 
development of atomic energy, adopt programs that will improve the environment, pay 
for space programs, and embrace recommended health practices. However, it is less clear 
what percentages of the population must be at higher levels of scientific understanding 
in order for individuals to enjoy satisfying lives and for an economy to compete and 
prosper. 

In order to make the results of the science assessment more understandable to 
policymakers and taxpayers who must make these judgments, the project has defined or 
"anchored" five points or levels on the science proficiency scale — (300, 400, 500, 600, 
and 700) — in terms of what students who perform at these levels know and can do 
related to science. These points or levels are also illustrated by sample questions chosen 
from the assessment to represent the kinds of tasks that students at each level typically 
can address successfully. The five levels and sample items are presented in FIGURE 4.2. 
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LEVEL Know Everyday Selene* Farts 

300 Students at this level know some general scientific facts of 

the type that can be learned from everyday experiences. For 

example, they exhibit some rudimentary knowledge concerning the 
environment and animals. 

LEVEL Understand and Apply 

Sim P |e Scientific Principles 

400 

Students at this level exhibit a growing knowledge in the 

Life Sciences, particularly human biological systems, and can apply 
some basic Principles from the Physical Sciences including force. They 
also display a beginning understanding of some of the basic methods 
of reasoning used in science, including classification and interpretation 
of statements. 



U ypu throw each of ibe following jw.iy after ,i picnic, which will decay 
the fastest and not harm the environment 

d> An appte core O A plasiic hnttlo 

CE> A metal no Ft -drink can CD A jtta^s hot lie 

Which of the following is NOT a reflex action^ 

d> Outekly closing your eyelid when something i& a |>out to hi*. \ n (mr eye 
<S> Falling over or tripping on a stone 

<2> Pulling away your hin^ when you accidentally touch a hot iron 
O I or king yogr leg when the Jocior taps your knee 



Use Scientific Procedures 
and Analyze Scientific Data 



LEVEL 

500 

Students at this level have a grasP of experimental proce- 
dures u;>ed in science, such as designing experiments, controlling 
variables, and using equipment. They can identify the best conclusions 
drawn from data on a graph and the best explanation for observed 
phenomena. Students also understand some concepts in a variety of 
science content areas, including the Life Sconces, Physical Sciences, 
and Earth and SPace Sciences. 



A teacher telt a plant in a duV, d.i^room during the school's ten da V 
spring break. She placed a light near the plant, and she watered *.he plant 
well. When students returned to school iftcr spring break, what do you 
ihin); the plant looked l'^ c,h Fill >n the oval u nder the picture you choose. 




Q> I don't knot/. 



LEVEL Understand and Apply Inlermediale 

800 Sc ' enimc Know * ei ^ e amf PrJnci r>^s 

Students at this level demonstrate an understanding of 

intermediate scientific fads and Principles and can apply this under- 
standing in designing experiments and interpreting data TheV also can 
interpret figures and diagrams used to convey scientific intormatioa 
Students at this level can infer retatiunships and draw conclusions by 
applying facts and principles, particularly from the Physical Sciences. 

LEVEL Integrate Scientific Information 
and Experim&nlal Evidence 

700 

Students at this level can Interpret experimental data that 

involves several variables. They also can interrelate information ^Pre- 
sented in a variety of forms— text, graphs, figures, and diagrams. 
Students can make predictions based on data and observations and 
are aware of limitations of extrapolation. Students demonstrate a 
growing understanding of more advanced scientific knowledge and 
concepts, such as the definition of a calorie or the concept of chemical 
change 



Group A 

Water vaPor 

Oxygen 

Atr 



Group B 
Ice 

Aluminum 
Iron 



Croup C 

Alcohol 

Water 

Gasoline 



The substances above, each at room temperature, ^ avC been classified 
into groups. On what Property is the classification ba£exl- 

O Chemical composition 

d> Specific heai 

CD State oi matter 

Abundance within the Earth's crust 



A child sits at ihe end oJ a seesaw 6 meters long The balance Poun is in 
the middle of the seesaw. Tt nass of the child is 25 kilograms. If your 
mass >s £0 kilograms and yflL rfjsh to balance the seesaw, at approxi- 
mately what distance from thehalance point would you need to sit on 
the oppc;ite side? 

G> O 5 mct«r d> 1 5 meters O 2.0 meters 3 O meters 
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The Power of the Scale 

Averages or means often hide important information. The availability of the five 
descriptive scale points permits a look at the percentages of students from each popula- 
tion that have acquired the knowledge and skills reflected by each of the defined levels 
{TABLE 41), 






Know 
Everyday 
Facts 


Apply 
Simple 
Prrnciples 


Analyze 
Experiments 


Apply 
Intermediate 
Principles 


Integrate 
Experimental 
Evidence 


LEVEL * 


300 


400 


500 


600 


700 


British Columbia 


100 


95 


72 


31 


4 


Korea 


100 


93 


73 


33 


2 


United Kingdom 


98 


89 


59 


21 


2 


Quebec (English) 


99 


92 


57 


15 


1 


Ontario (English) 


99 


91 


56 


17 


2 


Quebec (French) 


100 


91 


56 


15 




New Brunswick (English) 


99 


90 


55 


15 




Spain 


99 


88 


53 


12 




United States 


96 


78 


42 


12 




Ireland 


96 


76 


37 


9 




Ontario (French) 


98 


79 


35 


6 


<1 


New Brunswick (French) 


98 


78 


35 


7 


<1 


*Jackknifed standard errors for percentages range from less than .1 1o 2.6 and are provided in She Data Appendix. 



An examination of the data displayed in Table 4.1 is revealing in two ways: first, it 
demonstrates possible attainment levels and secondly, it illustrates the diversity cf 
achievement across populations. These data, for example, show that the 13-year-olds in 
British Columbia as a group have acquired science knowledge and skills that are far 
superior to those of their counterparts in several other populations. The question arises 
concerning what is required for other countries and provinces to evidence similar 
success. 

Obviously, there are a host of factors to be considered. Some, such as the school 
curriculum, the time devoted to science instruction, and the types of typical classroom 
activities, can be altered by the educational system. Other variables, such as socioeco- 
nomic conditions, the level of parents' education, and the societal value placed on the 
study of science, are largely beyond the power of the schools to alter 
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Nonetheless* it is a challenge that 95 percent of British Columbia's 13-year-olds can 
apply simple scientific principles (Level 400), while less than 80 percent of those in the 
United States, Ireland, Ontario (French), and New Brunswick (French) demonstrate the 
same level of competence. The performance of the other poputetiuns is in between these 
two extremes. 

Two countries, British Columbia and Korea, stand out. with more than 70 percent of 
their 13-year-olds able to use scientific procedures and analyze data (Leve! 500, the 
midpoint on theproficiencyscaleX Six other countries and provinces can claim that more 
than one-half of their students have achieved thesame competencies. The remaining four 
populations, the United States, Ireland, Ontario (French) and New Brunswick (French), 
are at the lowest end of the scale, with only about 35 to 40 percent of their students at 
Level 500 or above. 

Finally, more than 30 percent of British Columbia's and Koreas 13-year-olds can apply 
intermediate scientific knowledge and principles (Level 600), while fewer than one out 
of 10 of their peers in Ireland, Ontario (French), and New Brunswick (French) are able to 
do so. 

The large ranges of findings at performance Levels 400 (19 percentage points), 500 (38 
percentage points), and 600 (27 percentage points) certainly demand consideration. 
Moreover, increased percentages of students performing at the higher levels seem 
essential if an economy is to remain healthy and grow in today's competitive technolog- 
ical environment Cultural values as well as educational practices must be examined if 
improvements are to be made. 
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The Gender Gap 

Approximately 50 percent of the population of 13-year-olds in every country and 
province is female. Consciously developing the knowledge and skills of young worien 
represents a decision that can profoundly affect a country s economic achievement. There 
are far greater discrepancies between boys* and girls' performance in science than in 
mathematics (FIGURE 4 + 3). Males outperform females sigmitcantly in all populations 
except those of the United Kingdom and the United States. The greatest difference is in 
Korea, where males outperform females by nearly 40 scale points. 
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_[ MALES 
1 FEMALES 



Statistically significant tMference 
between groups at the 0;> level. 




MALE 
FEMALE 



BRJTtSH 


KOREA 


UNITED 

kingdom 


OUEBEC 
IEHGUSH) 


ontaric 

(ENOJSH) 


QUEBEC 
{ffi£HCH| 


HEW 
BRUNSWICK 
lEHGUSHlt 


SPAIN 


mm 

STATES 


IRELAND 


ONTARIO 
IFREHCHI 


NEW 
BRUNSWICK 
IFRENCK) 


562 3 


567.5 


524J 


525.4 


524.3 


523.7 


517.2 


518.0 


481.9 


460.4 


474,9 


477.4 




(3.6) 


(5 3) 


(3.8) 


(3.8) 


(4.1) 


(4.0) 


(4.S) 


(6.1) 


(5.2) 


(3.1) 


(5.0} 


541.6 


530.6 


514.9 


505.7 


504.7 


502.1 


503.3 


489.5 


474.9 


456.5 


463.1 


450.2 


(2.3) 


(4.1) 


13.3} 


(3.1) 


(3.0) 


(3,6) 


(41) 


(5 0) 


(4 7) 


(3.8) 


(2.7) 


(4.3) 



* Jackkntfed standard errors are presented in parentheses. 

t Background data are missing from 32% ol ihe New Brynw- * * . :n) %\vf n:s. 7ne resultant effect on ihe differences between groups 
for this particular province ts estimated to be less t' ~ raid d error. 



Where Bo We Stand? Are the Results Gctod Enough? 



The answer to the first query is clean but the challenge of the second is more troubling, 
It may be comforting to learn that close to 100 percent of today s 13-year-olds in all of the 
populations assessed know some everyday science facts. But what degree of scientific-rea- 
soning skills must be developed to enable citizens to improve their lives and to supply the 
effective workers T scientists, and researchers necessary for the year 2000? The hope is that 
these results will enlighten the debate necessary for each country and province to reach 
consensus on these important issues. 
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PART II 

SCIENCE 



CHAPTER FIVE 

Science: Instruction 
and Attitudes 



T 

Jit is reasonable to assume that parent student and teacher behaviors affect learning. 
A supportive parent, an attentive child, and acreative teacher would seem to he the ideal 
combination for success. Identifying t!:e specific measurable behaviors that reflect these 
characteristics has been a challenge to researchers over the years. Survey results, as in 
this study, cannot establish causal links between such factors and student achievement 
However, data on classroom practices, homework, support from home, and student 
attitudes can illustrate important differences in participating countries and provinces 
and provide a context for understanding their achievement results. 17 



Learning Science 

Classroom Activities, Included in the Questions about students backgrounds was a 
series asking them about their science classroom experiences. The results are high- 
lighted in TABLE 5.1. Entries reflect the percentage of students reporting frequent 
activity* "almost every day" or "several times a week," of the types described. Data are not 
provided for the other options, "once a week/' "less than once a week," and "never." 
Frequency of these activities is not consistently related (positively or negatively) with 
performance among participating groups, so performance data are not presented. 



,: In reading this chapter, note that background data arc missing from ?>2\ of the Nlv Brunswick lEnftiish} students. The 
resultant effect on the difference* amonfi ftnmps for this particular province is estimated to he Itrss than one standard error, 
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Watch a Mm 




Oo 






Read 


Solve 


or television 


Walth 


experiments 


Oe> 




your 


written 


program 


the teacher 


with 


experiments 




science 


science 


about 


do 


other 


by 




textbook 


problems 


science 


experiments 


students 


yourseM 


British Columbia 


70 (1-4) 


59 (1.2) 


10 (0.9) 


47 (1,5) 


39 (1.5) 


23 (1.2) 


Korea 


63 (1,2) 


32 (1,4) 


23 (0.9) 


47 (1.4) 


27 (17) 


9 (0,6) 


Uniled Kingdom 


23 (1.4) 


31 (1.4) 


4 (0.4) 


38 (17) 


57 (1.9) 


38 (1.6) 


Quebec (English) 


52 (1,8) 


52 (1,3) 


9 (0.8) 


32 (1.5) 


31 (1-6) 


21 (12) 


Inlario (English) 


31 (2.1) 


37 (1,3) 


11 (1.4) 


26 (1.8) 


23 (2.2) 


18 (1,6) 


Quebec (French) 


45 (14) 


62 (2.0) 


17 (1.2) 


38 (1.6) 


31 (19) 


17 (10) 


New Brunswick (English) 


60 (17) 


48 (16) 


8 (07) 


25 (1.8) 


10 (0.8) 


8 (07) 


Spain 


66 (2.0) 


66 (2,0) 


23 (1,5) 


51 (1.5) 


16 (1.5) 


15 (13) 


United Slates 


70 (1.8) 


54 (2.0) 


14 (1,6) 


19 (1.5) 


16 (1.9) 


12 (1,1) 


Ireland 


56 (2.0) 


28 (14) 


8 (0.8) 


35 (1.9) 


14 (1.4) 


10 (1.0) 


Ontario (French) 


20 (1.1) 


48 {1-6} 


17 (1.2) 


29 1.4) 


3o (1,6) 


15 (1,0) 


New Brunswick (French) 


44 (1.9) 


60 (2,2) 


17 (1.4) 


30 (1.4) 


20 (13) 


10 (0.9) 


"Jackkmfed standard errors are presented m parentheses. 



In 7 out of 12 of the populations, most students (50% to 70%) report reading from their 
science textbook several times a week About 45 percent of the students in Quebec 
(French) and New Brunswick (French) and fewe*- students (20% to 30%) in the United 
Kingdom and Ontario (English and French) say they read their textbooks that often. 
About one-third to two-thirds of the students in all populations regularly spend class- 
room time solving written problems. Except in Korea and Spain, few students say the) 1 
regularly watch science films or television programs in class. About one-quarter of the 
Korean and Spanish students report viewing science programs several times a week or 
more. 

Textbook reading, working on written problems, and viewing films or television 
programs are essentially passive activities. In order to learn higher-level concepts, science 
educators often recommend that students become involved in using scientific procedures 
by conducting experiments. However, teachers may face a number of barriers in this 
regard: lack of equipment lack of laboratory space, or large class sizes. 

In British Columbia. Korea, and Spain, students indicate that experimentation is often 
demonstrated by the teacher. About 50 percent of the students from these populations 
say they watch their teachers do experiments on a regular basis. Other populations do so 
less frequently. Almost 60 percent of the students from the United Kingdom and more 
than 30 percent of the students from British Columbia, and Quebec (English and French) 
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report doing experiments with other students several times a week or more. Other 
populations report lower frequency of this activity. Fewer students do experiments on 
their own; generally 10 to 20 percent of the students report regular activity of this sort. 
Again, more students (38%) from the United Kingdom report regularly doing experi- 
ments on their own. 

Students in the United Kingdom seem to be the most involved with hands-on experi- 
ments 35 demonstrated by teachers, with others, or by themselves, suggesting a priority 
among their teachers for this kind of activity, British Columbia's students also are 
regularly involved with all types of experiments. These two populations performed well 
on the 1AEP multiple-choice items. However, they might have had greater opportunities 
to demonstrate their knowledge and skills on a performance-based test. The young 
people in New Brunswick (English) and the United States are the least involved in 
experimentation on a regular basis. 

Homework. Students were also asked about science homework. Generally, students 
in all participating populations say they spend little time doing science homework 
(TABLE 5,2), Except in Spain, between one- half and three-quarters of the students report 
spending less than one hour per week on science homework. In Spain, 42 percent of the 
13-year-olds report doing one to two hours of science homework a week and 19 percent 
report doing three hours or more weekly. 




^ Percentages Reporting Amounts of \ . ^ 
MWfeeklyiScienceiHomewof^ge^131^aj <T^>mmr 





Less than 1 hour 


1 to 2 hours 


3 or more hours 


British Columbia 


51 (1.4} 


37 (12) 


12 (0 8) 


Korea 


57 (1.1) 


36 (11) 


7 (0.5) 


United Kingdom 


60 (17} 


35 (14) 


5 (0.6) 


Quebec (English) 


63 (14} 


30 (12) 


7 (07) 


Ontario (English) 


72 (1,3) 


24 (10) 


4 (0.6) 


Quebec (French) 


69 (1.4} 


26 (13) 


5 (0.6) 


New Brunswick (English) 


62 (1.1} 


29 (0,8) 


8 (C8) 


Spain 


39 (18) 


42 (16) 


19 (13) 


United States 


66 (18} 


26 (16) 


8 (10) 


Ireland 


70 (1.3} 


25 (12) 


6 (0.5) 


Ontario (French) 


69 (14) 


24 (11) 


7 (0 7) 


New Brunswick (French) 


68 (17) 


23 (14) 


9 (0,9) 


'JaCkkmfed standard errors are Presented in Parentheses. Percentages do n °t always sum to 100 due to rounding. 
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As is the case in mathematics, the relationship between the amount of time spent on 
science homework and science proficiency is not consistent across all the populations 
(FIGURE 5 + l) + However, for the three top performers, British Columbia. Korea, and the 
United Kingdom, within each population the more time reported spent on science 
homework, the higher the science achievement. 
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Home Involvement 

Today, in most countries, science concepts are not learned exclusively in the classroom. 
Often parental interest in science supports or enhances school learning. Students were 
given a list of science activities and were asked whether or not someone at home ever 
engaged in these activities with them. The results are presented in TABLE 5.3. 
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Talk aboul 




Watch 


Help you 


Help you 




urh^t iron am 
Wriol |uU ale 


i din 


d ablElll*E 


with 

Willi 


work 




learning 


about 


program 


your 


on a 




m science 


science 


on 


science 


science 




class 


topics 


television 


homework 


pro [eel 


British Columbia 


55 (1.1) 


40 (1.1) 


33 (1.2} 


49 (15) 


51 (1,3) 


Korea 


ro M 1 ^ 

53 (i i) 


31 P'l) 


It \ \ . i ) 




00 M OX 
00 


United Kingdom 


57 (1.3) 


39 (1-3) 


35 (1.3) 


50 (1.2) 


39 (1.4) 


Quebec (English) 


48 (1.4) 


41 (1.0) 


47 (1.1) 


30 (1,4) 


38 (1,5) 


Ontario (English) 


44 (1,4) 


42 (1.6) 


40 (1.2} 


38 (1.4) 


62 {1.7} 


Quebec (French) 


54 {1,2} 


47 (1.2) 


50 (1 4) 


31 (1.4) 


34 (1,3) 


New Brunswick (English) 


49 (1,7) 


38 (1,3) 


42 (1.2) 


41 (1.5) 


58 (15) 


Spain 


57 (1,5) 


40 (1,6) 


58 (17) 


33 (1.5) 


43 (1,5) 


United States 


53 (2,1) 


42 (2.6) 


37 (2,8) 


47 (1,9) 


56 (2.2) 


Ireland 


47 (1,3) 


38 (1,5) 


. 50 (1,2) 


25 (1,1) 


24 (1.1) 


Ontario (French) 


46 (1,2) 


47 (1,5) 


41 (1.2) 


37 (1.1) 


60 (1.3) 


New Brunswick (French) 


"44 (1.6} 


41 (1,4} 


49 (1,8) 


30 (1,4) 


45 (1^3) 



" JackknifecJ siandard errors are presented in parentheses. 



In most populations, between 30 and 60 percent of the students mport that someone 
at home asks about their science work in school, talks about scientific topics at home, or 
watches science programs on television. Watching science programs is very prevalent in 
Korea; more than 70 percent of Korean students report this kind of home activity. Korea 
has extensive, high-quality educational programming, especially in th<» sciences, and this 
is probably reflected in its students' viewing habits. Also Korean educators note that 
while the average education level of their students 1 parents is lower than that of parents 
from other participating populations* Korean parents are very supportive of their chil- 
dren's intellectual development. 

Generally, there is a positive relationship between the level of home involvement in 
science activities and student achievement (FIGURE 5,2), In all 12 populations, students 
who report home involvement in all three activities are the highest .science performers, 
while students who experience no home involvement fare least well on the assessment. 



Home involvement aiso takes the form of help with homework and science projects. 
The amount of this kind of home involvement varies from population to population. 
Fewer Korean students report this type of help. The relationship between this type of 
home involvement and achievement tends to be negative, suggesting that help from 
home on schoolwork may be forthcoming more often when students are doing poorly in 
school. 
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* Jackknifed standard errors fer proficiencies range Irom 2,7 to 85 and are provided in ihe Data Appends. 



Student Attitudes Toward Scaence 

Student attitudes toward science were also assessed and results are presented in 
TABLE 5-4. The table indicates the percentages of students who say they "strongly agree" 
or "agree" with statements about the utility and importance of science or who indicate 
they like science "a lot" or "a little/' 

Most students, between 50 and 85 percent in all populationsagree that learning about 
science is useful in everyday life and important for getting a good job. Atso, between 65 
and 85 percent of the students say they like science, and these students generally are 
higher performers than their peers who dislike science. 
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Much of what you 


It is important to 


r like 




learn in science 


know some science 


science a 




classes is useful 


in order to get 


Utile or 




in everyday life. 


a good lob. 


a lot. 


British Columbia 


52 (1.2) 


72 (0.9} 


72 (0.9) 


Korea 


82 (0.8) 


64 (1,2) 


82 (10) 


United Kingdom 


63 (1.1) 


80 (10) 


82 (10) 


Quebec (English) 


48 (1.2) 


59 (1.3) 


66 (12) 


Ontario (English) 


55 (1.7) 


77 (1,1) 


74 (10) 


Quebec (French) 


56 (13) 


62 (12) 


78 (0,9) 


New Brunswick {English) 


55 (1-5) 


74 (1.2) 


68 (14) 


Spain 


78 (1.6) 


59 (1.5) 


73 (16) 


United States 


50 (2.4) 


70 (1.9) 


68 (2,1) 


Ireland 


64 (1.5) 


66 (1.2) 


72 (15) 


Ontario (French) 


67 (1.3) 


85 (0.9) 


84 (0,9) 


New Brunswick (French) 


62 (1.5) 


72 (14) 


70 (15) 


'Jackkniled standard errors are presented jn parentheses. 



Most countries and provinces could probably benefit from supporting positive atti- 
tudes about science and promoting its value in today's modern societies. Generating 
interest among parent groups and the general population about the content and the 
methods used by scientists might encourage young minds to pursue science-related 
school work and hobbies with greater enthusiasm and therefore with greater effect. 
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PART II [ 



SCIENCE 



CHAPTER SIX 

Science: Topics 

T 

JLhe concepts introduced in chapter three concerning the "intended curriculum" 
reflected in curriculum guides and textbooks, the ''implemented curricuiurrT or the 
actual practices in schools and classrooms, and the "achieved curriculum" usually 
measured by student achievement are also relevant to the present discussion of various 
science topics. This chapter presents achievement results for five science categories and 
summarizes teachers ratings of students opportunities to learn the concepts assessed 
within these content areas. 

The 54 science questions included in the final analysis of the 1AEP assessment were 
divided into five topics as indicated in TABLE 6.1. 18 The results that follow present 
average percents correct for the items within each content area. 19 The proficiency scale 
summarizes overall performance in science. The average percents correct describe 
achievement in content areas often included in the science curriculum. Within each topic 
there are items that reflect many different levels of the proficiency scale from low to high. 

As indicated in the discussion of mathematics topics results, it is appropriate to 
compare the average percents correct among countries and provinces for the same groups 
of questions, but it is nonappropriate to compare the average for one topic with that of 
another within the same country or province. Also, because the number of questions 
within each category is relatively small, the results obviously do not represent a compre- 
hensive assessment of each topic. 

tH ln the analysis six Questions were droPPcd from the Gfl-item science assessment because cif diitercntirn' item functioning, to 
leave a total of 54. 

'"The number of Questions within each topic wa& insufficient for devchpiriti (oPic-rclated Proficiency scaled 
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STAB tE ; 0i^lS«liil| 

^Number^of|Sc]enGefitem#yiropicM 




Lite 
Sciences 



Physics 



Chemistry 



Earth and 
Space Sciences 



Kalure 
ol Science 



Total 



14 



10 



14 



54 



'In the analysis, six questions were dropped from Ihe 6Chtem science assessmenl because of drflerennai iiem luncuomng. lo leave a lotat 
Of 54 
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from A to 1.0 and are provided in the Data Appendix. 
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Li fe Sciences 

This topic focuses on plants and animals, 
including the interdependence of living 
things, characteristics of different species, 
photosynthesis, growth and adaptation, and 
ecology. Also covered are characteristics of 
human biological systems. The relative per- 
formance of countries and provinces in Life 
Sciences mirrors their achievement in sci- 
ence overall, except for Quebec ( French ) f 
where students perform relatively higher in 
this topic than they do overall. The average 
percent^ correct for Life Sciences are pre- 
sented in FIGURE 6.L 30 



^'For this arid subsequent analysis of achievement by topic, pi filiations are cite J as deviating from their normal pattern if the 
difference between their deviation from the mean for t he topic and their deviation from the overall mean is greater than twice 
the standard error o.' the difference between these deviations. The standard error was taken lobe euual to I hi composite value 
of 1,35. 
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Physics 

Questions in this category cover the con- 
cepts of force, distance, weight, volume, and 
acceleration. Also assessed are simple optics, 
such as mirrors and lenses, and very basic 
understandings of electricity. All popula- 
tions perform in Physics about as well as 
they do in science overall, except New 
Brunswick {French) students, who perform 
at higher levels, and British Columbia stu- 
dents, who perform at lower levels than they 
do overall {FIGURE &2) + 



Chemistry 

Questions in this content area cover states 
of matter, the nature of solutions, reactions 
of matter, and very basic understandings of 
the atom. The range of achievement of par- 
ticipating countries and provinces is greater 
in Chemistry than in any other topic (FIG- 
URE 6,3), Students from British Columbia 
and Korea have higher averages in Chem- 
istry than they do for science in general, and 
students from the United Kingdom and Que- 
bec (English) have lower averages than they 
do overall. Other populations perform about 
the same as they do for a!l science Items, 
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* Jackknifed standard errors for proficiencies range 
Irom A lo .9 and are provided in the Data Appendix, 
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* Jackknifed standard errors for proficiencies range 
from 5 to 1 .0 and are provided in the Dala Appendix. 
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Earth and Space Sciences 

This topic includes the earth's history, the 
earth's atmosphere, and physical aspects of 
the earth's surface. Also included are ques- 
tions on the solar system and space explo- 
ration. Again, the pattern of performance in 
this area generally mirrors overall science 
achievement (FIGURE 6 + 4) + Lxceptions are 
Quebec (French) and New Brunswick 
(French) where students perform relatively 
lower in this topic when compared with 
their performance overall 
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* JaekkflUed standard errors tor proficiencies range 
from A to 1.0 and are provided in the Data Appendix. 
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NaEare of Science 

This area assesses understanding of scien- 
tific methods irrespective of the content 
area. It includes questions on logic, testing 
hypotheses, using scientific equipment, 
designing experiments, and interpreting 
results. Korean students perform relatively 
lower in Nature of Science when compared 
with their overall high science achievement 
level (FIGURE 6,5), Achievement of other 
populations in this topic is at about the s?me 
level as their overall performance. 



52 



51 



Opportunity to Learn Science hy Topic 



Following the same procedures as those for mathematics, a teacher or science coordi- 
nator in each participating school was asked to indicate the percentage of seventh- and 
eighth-grade students who had already had an opportunity to learn the concepts tested 
by each question in the science assessment. 21 The analysis of these data focuses exclu- 
sively on the results from schools in which more than 75 percent of the students had 
already had an opportunity to leam the content assessed. It was assumed that students 
in these schools had been "exposed' 1 to the content. The ratings were weighted by the 
proportion of 13-year-olds in each of these grades and summarized across items in each 
topic area. This provided an estimate of the percentage of questions to which an average 
13-year-old h d been exposed for each content area, 22 

FIGURE 6.6 combines the achievement information on science topics presented earlier 
with the opportunity-to-learn ratings. The average percents correct (the bars) indicate 
the percentage of items in a topic that students on average have answered correctly, The 
average opportunity-to-learn ratings (thedots) indicate the percentage of items in a topic 
to which students on average have been exposed. When the dot and its confidence 
interval are within the bar, student performance is higher than teacher estimates of 
student exposure to the concepts tested by the items. 

In all populations except Spain, students generally perform better in science topics 
than their teachers' ratings would suggest. Spanish teachers tend to estimate their 
students' exposure to science topics higher than teachers from the other countries and 
provinces, and Spanish students tend to achieve about at the level expected by their 
teachers. 

Two factovs may help to explain the opportunity-to-learn results. The first is that the 
teachers completing the questionnaires may not know a great deal about the content 
coverage in prior grades. This is evident from the British Columbia results, in which the 
eighth-grade teachers in secondary schools consistently assigned lower opportunity-to- 
learn ratings than the seventh-grade teachers in the provinces elementary schools. 

A second reason for opportunity-to-learn ratings being iower than the students 
average percents correct may be because the science assessment tended to cover science 
concepts not taught exclusively in the school curriculum- Only about one-quarter of the 
science questions dealt with specific science knowledge. The remaining items assessed 
application and integration of science concepts, to which students may have been 
exposed outside of the school environment. 



?1 ln sume cases, rating were the rcsuEt <>f the consensus nf teachers at each tirade, jnd iri cases more than !mt ' tocher 
in a school Provided raLnfis and responses w<:re weighted aPProPriatelv, 

^Norircsponscrateof teachers translated into missinS data for more than 10 Percent of the student roPulatiun^ in In." hind. New 
Brunswick (English}, New Brunswick (French). Ontario (English Ontario (French), Quebec (English). Ouehcc (French), and 
the United States, 
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If bar terminates above the dot and confidence interval 
i; indicates that student performance is higner than 
teacher estimates cf student exposure to the concepis 
tested by the items. 
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PART II 

SCIENCE 



CHAPTER SEVEN 

Personal Learning Environments 



T 

JLn addition to specific information on mathematics and science, the assessment gath- 
ered general background information about television watching, homework and interest 
in school from each student participating in the study. Results identify similarities and 
differences across geographic areas and substantiate and sometimes challenge conven- 
tional wisdom concerning the relationships between these characteristics and student 
achievement 



Television Viewing 

Students were asked how much television they usually watch each day. The data in 
TABLE 7,1 indicates that the norm is two hours or less of viewing daily for Korea. Quebec 
(French), Spain, and Ireland, and three to four hours daily for the remaining populations. 
Smaller percentages of students, from 7 percent in Korea to 31 percent in the United 
States* report spending five or more hours in front of the television set 

FIGURE 7.1 shows that when these findings are related to performance, for every 
grouP. the greater the amount of time spent watching television per day. the poorer the 
performance in science* A similar relationship holes for mathematics. 





2 hours or less 


3 to 4 hours 


5 or more hours 


British Columbia 


41 (1.0) 


43 (1.0} 


17 (0,9) 


Korea 


49 (1 + 5) 


44 (1,3) 


7 (0,6) 


United Kingdom 


28 (1.2) 


45 (1.1) 


27 (1.3) 


Quebec (English) 


37 (1.1) 


44 (1.0) 


19 (0.8) 


Ontario (English) 


35 (1,5) 


43 (1,2} 


22 (1.1) 


Quebec (French) 


49 (1,4) 


40 (1,1) 


11 (0.8) 


Mew Brunswick (English) 


29 (1.3) 


49 (1.5) 


22 (1.0) 


Spain 


46 (2.3) 


41 (1.7) 


13 (1,1) 


United States 


27 (1.4) 


42 (17) 


31 (1.6) 


Ireland 


45 (17) 


41 (14) 


14 (1,2) 


Ontario (French) 


33 (1.1) 


46 (1.1) 


21 (1,2) 


Hew Brunswick (French) 


31 (17) 


43 (1.5) 


22 (1,4) 


"Students are those who have a science profacrency score Jackknifed standard errors are presented in parentheses Percentages do not 
always sum to 1 00 due lo rounding. 
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* Jackkniled siandard errors lor proficieticies iange from 2.7 to 7.4 and are pmvided in ihe Daia Appendix. 



Homework 

Students were also asked how much time they usually spend on homework each day 
for all school subjects combined, and the options were 1 don't usually have homework 
assigned/' "I have homework but f don't usually do it.* s, l/2 hour or less " i% \ hour/ 4 "2 
hours/' and **more than 2 hours." 

Except for Spain and Ireland, most 13-year-olds (more than 50%) in participating 
countries and provinces report spending one hour or less daily on their school assign- 
ments (TABLE 721 More than 50 percent of the Spanish and Irish students say they do 
two or more hours of homework each day. 
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None 


Don't 


1 hour 


2 or more 




assigned 


doit 


or less 


hours 


British Columbia 


4 (0.5) 


4 (0,5) 


66 (1.2) 


26 (1.1) 


Korea 


2 (0,3) 


3 (0.3) 


68 (1.1) 


23 (1.2) 


United Kingdom 


8 (12) 


2 (0.3) 


54 (1.6) 


35 (1.6) 


Quebec (English) 


2 (0.5) 


3 (0^4) 


62 (L4) 


33 (1.2) 


Ontario (English) 


5 (0.5) 


3 (0.4) 


68 (12) 


25 (1.2) 


Quebec (French) 


1 (0.2) 


1 (0.3) 


67 (2.0) 


31 (2.0) 


New Brunswick (English) 


3 (0.5) 


5 (0.6) 


74 (1.4) 


19 (1.3) 


Spain 


1 (03) 


1 (0.3) 


39 (1.7) 


59 (17) 


United States 


4 (0.9) 


5 (0.8) 


65 (2.4) 


27 (2.0) 


Ireland 


1 (0.1) 


2 (0.6) 


41 (1,6) 


57 (1.8) 


Ontario (French) 


6 (07) 


2 (0,4) 


69 (1,4) 


24 (1.3) 


New Brunswick (French) 


3 (07) 


1 (0^3) 


72 (1,6) 


24 (1.5) 



'Students are those who have a science Proficiency score. Jackknifed standard errors are Presented m Parentheses. Percentages tfo not 
awayssum to too due to rounding. 



When homework results are related to science performance, relationships tend to be 
confusing. In several populations the highest achievers are students who report doing 
two hours or more of homework daily. In others, the highest are those who say they spend 
one hour or less daily; and in one group al! students perform roughly at the same level 
including those who have no homework assigned and those who have homework but do 
not do it The relationship between homework and mathematics performance is equally 
confusing. Reasons for this inconsistency may be the same as those identified in earlier 
chapters. Homework may be assigned differentially; sometimes enrichment assignments 
are given to better students, and sometimes remedial work is assigned to poorer students. 
Also, some students may finish assignments quickly during the school day, while others 
may take more time at home. 

Students were also asked how often someone from home helped them with their 
homework: "almost every day/' "once or twice a week/' + 'once or twice a month," "never 
or hardly ever " or "don't have homework/' Their responses are summarized in TABLE 
7.3. Except in the United States, where help is more prevalent only about 25 to 35 
percent of students say they receive help once or twice a week. As seen earlier in similar 
mathematics and science-related questions, results suggest that poorer students are 
more likely to get regular attention at home. 



ERJC 




Pffttfff 







unci; ur iwtCB 


jiETcr or 


uun i n<iYc 




a week 


a monlli 


hardly ever 


homework 


Dntts ii Loiumoia 


00 (l.U) 


on in o\ 


4i (lu) 


1 (0.2) 


Korea 


22 (1.0) 


17 (1.0) 


61 (12) 


1 (0.1) 


United Kingdom 


38 (1.2) 


24 (11) 


36 (12) 


2 (0.5) 


Quebec (English) 


27 (1.2) 


20 (0.8) 


53 (12) 


1 (0.2} 


Ontario (English) 


31 (1.3) 


22 (10) 


46 (13) 


1 (0.2) 


Quebec (French) 


32 (1-2) 


15 (0,9) 


52 (12) 


1 (0.2) 


New Brunswick (English) 


32 (1.4) 


19 (10) 


48 (14) 


1 (0.3) 


Spain 


25 (1.4) 


11 (0.9) 


64 (17) 


1 (0.1) 


United States 


47 (2.1) 


14 (19) 


37 (27) 


2 (0.8) 


Ireland 


24 (1.2) 


14 (0,9) 


62 (15) 


1 (0.2) 


Ontario (French) 


36 (1.0) 


18 (0,9) 


44 (11) 


2 (0.4) 


New Brunswick (French) 


33 (1.4) 


15 (0,9) 


! 52 (141 


1 (0.2) 



"Sludents are those ttho have a scrence prolicrency score. Jackkmfed slandard errors are presented m Parentheses. Percentages do not 
always som to 100 due to rounding 



Interest in School! 

As reflected in many oth * studies and surveys, about 50 to 85 percent of the students 
in all countries say they like school {TABLE 7 A). Curiously students in populations that 
are typical of high performers (Korea) and low performers (Ontario-French) in science 
report being the most enthusiastic about attending school 



7he Pergonal Learning Environment 

All the factors described in this chapter, television viewing, homework, help with 
homework, and interest in school, help define the student s personal learning environ- 
ment Although teacher assignments and parental guidance influence student behavior, 
students themselves often make the decisions about how they spend their time outside 
of school and on what tasks they concentrate their attention and effort Television and 
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Like 
school 


Undecided 
a bom school 


Dislike 
school 


British Columbia 


64 (1,1) 


18 (0.8) 


19 (10) 


Korea 


85 (0.8) 


8 (0 7) 


8 (0.5) 


Uniled Kingdom 


72 (1.2} 


7 (07) 


15 (10) 


Quebec (English) 


61 (1.4) 


16 (10) 


23 (10) 


Ontario {English) 


66 (1.2) 


16 (10) 


18 (10) 


Quebec (French) 


76 (1.0) 


7 (0,5) 


17 (0.9) = 


New Brunswick (English) 


58 (17) 


14 (13) 


28 (12) 


Spain 


48 (2.3) 


39 (19) 


13 (1.4) 


Uniled States 


64 (2.0) 


14 (12) 


22 (19) 


Ireland 


61 (1.7) 


12 (0,8) 


27 (17) 


Ontario (French) 


78 (1.2) 


7 (0,6) 


15 (10) 


New Brunswick (French) 


69 (1.7) 


9 (0.8) 


23 (17) 


* Students are those who have a science proficiency score Jackkniled slandard errors are Presented >n parentheses Percentages do not 
always sum to 100 due to rounding. 



homework are often cited as competing for students 1 time and attention, and IAEP 
results suggest that television is the winner. Parents also make decisions about how they 
spend their own time> and these results suggest that mothers and fathers are more likely 
to spend some of their time helping their children with homework if the children are 
lower-performing students. Although creating an environment that encourages children 
to like school may not guarantee higher performance, it seems likely that parental and 
student involvement in meaningful educational activities is key to creating a personal 
environment conducive to learning. 
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- PART III - 

Context and 
Commentary 



In the United States there are 3.000,000 young men and women aged 13. In New 
Brunswick, Canada, there are only about 10.000. In many ways the diversity of the smaller 
group is probably as great as that of the larger, but the similarities are equally significant. 
While there are more subcultures reflected in one and supposedly more homogeneity in 
the other, anyone who has visited schools around the world will remember the feelings 
of familiarity in an eighth-grade classroom in SeviUa or Seoul or in a teachers lounge in 
Dublin or Detroit 

In order to provide a context for results, IAEP asked educators from each participating 
country or province to describe their social and educational environments and to 
comment on their results, 2 * For example, they reported the age at which their students 
typically begin school, the length of the school year, and the average class size. They also 
commented on the status of educational issues in the participating locations and the 
current emphasis (or lack thereof) on mathematics and science. 

Each country and province is in a unique situation with respect to its social and 
educational programs. Some are engaged in broad waves of educational reform. Others 
are focused almost entirely on grave economic concerns. Still others are consumed with 
political matters. 

This report will elicit different reactions from various segments of each political entity, 
In some cases, the story will be received with surprise, and in others, it will confirm 
existing perceptions. Conservative and progressive policymakers may view the same 
findings as indicative of very different problems. Our hope is that the data will be 
accepted as valid and reliable and viewed as useful for debates as well as supportive of 
positive changes. 



^These sections were written by individuals Participating in the project and reviewed by the tr organization a I, see acknowledg- 
ments The United Stales section was Provided by the staff of the Center for Assessment uf Educational Prioress ICAEP) at 
Educational Testing Service. Statistical data provided by each country and province .ire drawn from a variety of purees and in 
some cases may not bestricMy comnarabfe r T>ataon the number of 1 3* year-oJds a re the sums of weights derived from the survey 
results Because some schools and students were excluded from the assessment fur a variety of reasons these data may 
underre Present the true tot ft by t j P 10 five Percent, 
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Provincial 
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Population 


13-Year-Olds 
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School Year 
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23.4 (elem.) 




2,926,000 


504,000 


3^.000 


5 


185 


24,5 (sec.) 


Yes 



Contest 

British Columbia s educational system is characterized by centralized control over the 
development of curriculum goals, objectives, and the provision of resources. The elemen- 
tary core curriculum places considerable emphasis on mathematics and sciei :e. The 
Ministry of Education has traditionally taken a leadership role in establishing and 
maintaining educational standards through province-vvide testing and evaluation pro- 
grams. The results of evaluation studies and student achievement tests serve as useful 
indicators of system performance as well as fuel for discussions about public financial 
support for education. 

A major function of education is seen as the transmission of knowledge and values that 
society considers important. It is generally felt that the school system plays a major role 
in students' intellectual development^ lesser role in their vocational development, and 
shares responsibility with the family and other agencies for students' social 2nd human 
development. 

Approximately 95 percent of the students attend public schools. The major task of 
public schools is developing students ability to analyze critically, to reason and think 
independently, and to acquire basic learningskills and bodies of knowledge. Students are 
also expected to acquire a lifelong appreciation of learning, a curiosity about the world 
around them, and a capacity for creative thought and expression. The principal indicator 
of whether or not the system is meeting this goal is academic achievement, and schools 
are being held increasingly accountable by government and the public for performance. 

The public school system enrolls approximately 500,000 students, hasa teaching force 
of 27,000, and is organized into 75 school districts, that tend to be highly diverse both in 
terms of population size and geography. The province is characterized by both large- 
enrollment urban districts and small-enrollment rural districts, making the delivery of 
education services challenging. The average elementary school enrolls about 300 stu- 
dents, while typical secondary schools enroll about 800 students. 

Teachers are highly experienced, averaging nearly 15 years of service. As the teaching 
force ages, considerable pressure will be placed on universities to fill the widening gap 
between supply and demand. Further demands will be placed on the school system as a 
result of a recent increase in enrollment following a number of years of decline. Another 
issue of current interest is the provision of educational services to an increasing number 
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of children of immigrant families* many of whom are from Pacific Rim countries. This 
influx has placed heavy demands on schools to provide English-as-a-second-language 
instruction. 



Commentary 

Students performed very well on both the mathematics and science assessments, 
achieving higher average scores than most jurisdictions involved in the study. A high 
percentage of students was successful at the basic skills levels, and a significant percent- 
age reached proficiency levels in both subject areas that reflect mastery of higher-order, 
critical-thinking, and problem-solving abilities. 

Mathematics- Students ranked third in mathematics achievement. Up to and including 
Level 500, the performance of British Columbia's students was similar to that of the 
highest-ranking country (Korea). At Levels 600 and 700, the success of Korean students 
was nearly twice that of the province. 

Among the cognitive levels addressed in the 1AEP study, British Columbia's students, 
along with their Korean peers, were the highest performers on questions requiring 
problem solving. The acquisition of problem-solving skills is considered by the Ministry 
of Education to be of critical if iportance in the development of an educated person. 
Recent provincial assessments suggest that teachers have made concerted efforts to 
improve instruction in this area, which is emphasized in the new mathematics curricu- 
lum. 

Although scores were not low in Geometry and Algebra, these areas were identified as 
relative weaknesses. Previous assessments have identified Geometry as a weakness, and 
the mathematics curriculum introduced in 1988 has attempted to address this issue. 

Science. The achievement in science was exceptional with a ranking of first among all 
participating populations. Students scored consistently high on all content areas, 
excelling on questions requiring both knowledge and integration of scientific facts and 
principles, and especially on questions related to Chemistry. These findings are not 
surprising given the emphasis placed on problem solving and process learning in the 
elementary science curriculum. Recent assessments have found that teachers tend to 
integrate elementary science with other subjects, a fact that may help explain students 
high standing on problem-solving questions. The percentages of students performing at 
all five proficiency levels in British Columbia were similar to those for Korea. 

Opportunity to Learn. In sharp contrast to the students' strong performance, teachers 
indicated that students had little opportunity to learn the concepts tested in most of the 
science items and about half of the mathematics items. For many items, especially in 
mathematics, seventh-grade teachers indicated a greater opportunity to learn the con- 
cepts than did eighth-grade teachers. An explanation may lie in the fact that seventh- 
grade classes are typically found in elementary schools, while eighth-grade classes are in 
secondary schools- It appears that eighth-grade teachers are unfamiliar with the elemen- 
tary mathematics and science curricula. 
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Total 
Population 


School 
(K-12) 
Population 


Number of 
13-Year-Dlds 


Age 
School 
Begins 


Number of 
Days in 
School Year 


Average 
Class 
Size 


National 
Curriculum 


3,360.000 


861.000 


66.000 


4 


184 (elem>) 
130 (sec.) 


30,3 (eiem,) 


Yes 



Contest 

Eight grades of primary school serve children between the ages of 4 and 12. Although 
education is compulsory only between the ages of 6 and 15, about 60 percent of 
4-year-olds and 97 percent of 5-year-olds are enrolled in primary school. Over half a 
million children attend the 3,266 publicly aided schools in the country. A small number 
of pupils attend private primary schools. 

There are five or six grades in the four types of second-level schools: secondary, 
vocational, comprehensive, and community. Comprehensive and community schools are 
expected to provide a wide range of curricular offerings to their more than 300,000 
students. From the age of 6 to 14, practically all children attend school. At age 15, the 
participation decreases to 91 percent at 17 to 61 percent, and at 18 to 36 percent. 

Prior tol971,theprimary-sch^'>l curriculum was subject-centered, with heavy empha- 
sis on rote learning of arithmetir . d language skills. In line with a policy of Gaelirization, 
particular attention was paid to development of the Irish language and culture. In 1971, 
there was a change to a more child -centered curriculum, with more attention paid to 
individual differences and to discovery methods of learning. The curriculum guidelines 
laid down by the Department of Education address Religion, Irish, English, Mathematics, 
Art and Craft activities, Social and Environmental Studies, History, Civics, Geography, 
Music, and Physical Education. All primary-school teachers follow a common set of 
guidelines, but they have wide discretion in the choice of topics and the order in which 
they are presented. For second-level schools, the Department of Education prescribes 
curricula for a broad range of subjects that lead to public examinations — the Intermedi- 
ate Certificate after three or four years and the LeavingCertificate after a further two or 
three years. 

Since considerable official emphasis is placed on the Irish language, which is compul- 
sory in all primary and second-level schools, the amount of time available for other areas 
of study (e.g., Mathematics or Science) is obviously decreased. In Social and Environmen- 
tal Studies, children in primary school are introduced to the study of science mainly 
through Biology. In the final two years, pupils are introduced to basic concepts in the 
Physical Sciences. 

The traditional curriculum in secondary schools laid a heavy emphasis on the human- 
ities, Since the 1960s, there has been an effort to increase participation in scientific and 
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technical areas of study: This is reflected in the numbers of students taking courses and 
public examinations in these areas* although the numbers arc still not very large (e.g.* in 
1980, IS percent of students took Leaving Certificate Chemistry and 14 percent. Leaving 
Certificate Physics). Large classes and lack of facilities have hampered the full implemen- 
tation of the new curriculum and retarded innovation. 
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Commentary 

Mathematics. In mathematics Irish 13-year-olds performed* on average, better than 
their peers in the United States, about as well as students in the United Kingdom and 
Spain* but not as well as students in the Republic of Korea and most of the Canadian 
provinces. Like their peers in the other countries* most Irish students (98%) have 
mastered basic addition and subtraction tasks typical of Level 300. However* only 55 
percent of Irish students were able to perform the problem-solving tasks typical of Level 
500* compared to 78 percent of their Korean counterparts. They performed relatively well 
in !x>gic and Problem Solving and less we" in Data Organization and Interpretation, 
Teacher responses generally indicated that students have been taught most mathematics 
topics except for Geometry* in which students did much better than would have been 
expected on the basis of the low teacher ratings. 

Student self-reports confirmed that students spend a lot of time either listening to the 
teacher or working alone and relatively little time working in groups or with classmates. 
This is consistent with an instructional approach that places more emphasis on computa- 
tional skills and less on problem solving and other higher-level skills. The absence of a 
gender difference is significant and may reflect a movement over the last 25 years to 
improve the participation and attainment levels of girls in mathematics. 

Science. In science* Irish students lagged behind their peers in most of the other 
countries except the United States. While most (96%) have mastered everyday scientific 
facts (Level 300)* a much smaller percentage (37%) could analyze experiments (Level 
500). Teacher ratings of opportunity to learn indicated a fairly low level of exposure to 
science topics, particularly in the seventh grade, but more exposure to Chemistry and the 
Nature of Science than to Physics, Earth and Space Sciences and Life Sciences, 

Student self-reports of science classroom activity revealed a low level of practical work, 
both individually and with classmates. A disproportionate amount of time appears to be 
spent either reading textbooks or watching the teacher conduct experiments. Achieve- 
ment was significantly higher for boys than for girls on the science assessment asa whole. 
This is not surprising given the greater percentage of boys taking courses in nature 
science, but it is a cause for serious concern. 

Summary. In mathematics, these results illustrate the need for more attention to 
higher-level abilities than to routine computational skills. In science, there appears to be 
a strong case for the reexamination of its role in the curriculum, particularly with a view 
to increasing its attraction for girls. 
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Total 
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(K-12) 
Population 


Number of 
13-Year-OlrJs 


Age 
School 
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Number of 

Days in 
School Year 


Average 
Class 
Size 


National 
Curriculum 


40.432.000 


9.667.000 


903.000 


6 


220 1 


55 (cities) 
40 (rural) 


Yes 



The Republic of Korea is an increasingly industrialized nation with a growing economy 
guided by a series of five-year development plans under a highly centralized government. 
The population, which is homogeneous in both ethnic origins and language, is more than 
90 percent literate and growing at a slower pace than it- the 1950s. 

The Education Act of 1948 stipulates that the purpose of education is to "enable every 
citizen to perfect his personality, uphold the ideals of universal fraternity, develop a 
capability for self-support in life* and enable him to work for the development of a 
democratic state and for the common prosperity of all humankind/' 

In the 1970s reforms brought significant changes in curriculum and instructional 
techniques. The main thrust has been to develop an instructional system that draws not 
only on classroom lectures and the reading of textbooks but also on multiple learning 
materials and an extensive and very sophisticated set of television and radio programs. 
New learning materials and instructional techniques are being developed and field-tested 
regularly to discover more effective procedures. 

The current instructional system proceeds through five stages that the teacher uses in 
carrying out study units or lessons; a) planning, following the directions in the teachers' 
guide; b) diagnosing students' strengths and weaknesses by means of test items in a 
workbook; c) guiding student teaming by use of workbooks and television programs d) 
extending learning through the use of formative test items in workbooks; and e) 
evaluating the results of student learning with summative tests. 

Boys and girls ages 12 to 14 are generally enrolled in middle schools. Entrance 
examinations for these schools were abolished in 1969, and currently all applicants are 
assigned by computer to schools within a district Ninety-nine percent of students in this 
age group graduate from elementary school and move on to middle school which is free 
(tax-supported ). 

In middle schools, students study mathematics four hours per week during the first 
year and three to four hours per week during the second and third years. The same is true 
for science. There are generally 40 to 55 students in a classroom, and every student is 
assigned a permanent seat with teachers, rather than students, rotating for class changes 
Because of the class sizes, the lecture method is the rule in most classrooms, but in 
science, various experiments in laboratories are emphasized. 
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Commentary 



Mathematics. In mathematics, Korean students performed well above their peers in 
other countries and provinces in all topics except Logic and Problem Solving. They 
particularly excelled in Data Organization and Interpretation and Geometry* achieving 
well above the other populations in these two topics. This success may be attributed to 
textbook coverage of these two features of mathematics or emphasis by Korea's middle* 
school mathematics specialists. 

In comparing the results of students on the IEAP items with those from earlier 
international assessments, performance is better than in the past, perhaps reflecting the 
closer "fit" of this particular set of items to Korea's curriculum. It will be the responsibil- 
ity of the policymakers and educational leacic " to determine if the levels of achievement 
in the various topics are acceptable or if they should be improved. The fact that 40 per- 
cent of 13-year-olds could successfully perform tasks at Level 600 on the mathematics 
performance scale* nearly double the percentage of the other countries, is a source of 
satisfaction. 

Science. In science, Koreas 13-year-olds joined those from British Columbia as top 
performers across all five topics. Relative to their performance in science overall, students 
did better in Chemistry and worse in the Nature of Science. These results may reflect the 
patterns of the courses of study and the emphases of the curriculum. While it is gratifying 
to note that 33 percent of the 13-year-olds could successfully perform the tasks described 
as the ability to analyze scientific data (Level 600)* it may be that improvements should 
be considered as the technologies of the future are contemplated. 

Summary. Korea's success can be partially attributed to the nation'sand parents' strong 
interest in education, reflected in a 220-day school year. While there is virtually no adult 
illiteracy in the country* only 13 percent of Koreas parents have completed some 
postsecondary education. They nonetheless see education as the hope for their children 
and grandchildren. Everyone recognizes that Korea's job market is very demanding and 
the scientific and technological areas carry high prestige. 

Mathematics and science are areas of special interest and most middle schools and all 
secondary schools have specialists teaching these subjects While there is a mandated 
national curriculum* teachers are free to select one of five approved textbooks for most 
courses. Korea will continue to stress improvements in these subjects through research, 
support, and specifically through the current establishment of 14 special science high 
schools (one in each province). 
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Prouncial 
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712,300 


139.000 


10.000 


6 


182 


23.9 


Yes 



Content 



In 1967, the provincial government, took over the entire responsibility of public-school 
financing. The Schools Act vested in the Minister of Education the authority to prescribe 
school curriculum for all school districts. 

Having become an officially bilingual province (English and French) in 1969. the 
province recognized its linguistic duality in establishing, in 1974 two parallel but 
separate education systems. Subsequent*^ school boards and schools were established 
on a linguistic basis. There are now 15 Francophone school districts with 46,002 students 
and 27 Anglophone school districts with 92,052 students. At the junior-high level (grades 
7 to 9) ther^are 11,768 students (33%) in Francophone enrollments and 23,901 students 
(67%) in Anglophone enrollments. Each linguistic division of the Department of Educa- 
tion is responsible for its own curriculum. A structured science curriculum at the 
elementary level is a recent addition, The mathematics curriculum is currently undergo- 
ing transformation. 

Mathematics and science are taught from the first year of school. In elementary {grades 
1 to 6) and junior-high (grades 7 to 9) schools, teaching time in each subject is allocated 
as follows; 



Francophone Schools 



Grades 


1 and 2 


3lo6 


7 


8 and 9 


Mathematics 


300 min/week 


300 min/week 


280 min/week 


240 min/week 


Science 


70rnin/week 


90 min/week 


160 min/week 


160 min/Veek 



Anglophone Schools 



Grades 


1 Jo 3 


4 la 6 


710 9 


Mathematics 


300 min/week r 


300 min/week 


225 min/week 


Science 


60 min/week 


100 min/week 


180 min/week 



There are variations of schedules within schools. However, regardless of the form of 
the scheduling, the province stresses the importance of maintaining a balance between 
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all subject areas, All in all, there is strong interest at the provincial leve! in strengthening 
curriculum in mathematics and science, both at the elementary and at the junior-high 
levels, as well as in increasing the competence of teachers in those particular subjects. 



Commentary 

In general the results of the IAEP were consistent with the expectations of the 
Department of Education's curriculum and evaluation personnel With the exception of 
the results reported for t T ie French-speaking students in science, the relative perfor- 
mance of New Brunswick 13-year-olds on both the mathematics and the science assess- 
ments was anticipated. 

Mathematics. Revision and/or implementation of curriculum modification in New 
Brunswick is cyclical. Recently, a new and more relevant mathematics currieulr has 
been put into place in the elementary schools. The effect of this change is now be: 1 
at the junior-high-school level It is believed that if the 13-year-old boys and girls ho 
took the IAEP mathematics assessment had been taught mathematics under the new 
curriculum, their performance would have been higher. 

Science. Current curriculum and instructional practices also affected the performance 
of students on the science assessments. According to provincial curriculum guides, 
formal scipr^e instruction ranges from 60 minutes a week in the lower elementary grades 
to a maximum of 100 minutes per week by the end of grade six. However, in reality, the 
actual time devoted to science instruction is often much less, and science lessons are 
usually relegated to afternoons. Also, the quality of instruction appears to depend on the 
classroom teacher's own interest in and understanding of science. Very few elementary 
schools have science laboratories or other facilities related to the teaching and learning 
of science. Also. New Brunswick is a rural province lacking museums, planetarium^ and 
science centers, which are often helpful in fostering positive altitudes toward the 
learning of science. 

The lower results obtained by the 13-year-olds attending French-speaking schools can 
be attributed to a variety of factors, For example, a formal science curriculum guide for 
the elementary teachers and science in-service sessions for elementary and junior-high 
teachers have been available only recently. Also, new science textbooks have only 
recently been adopted for the elementary and junior-high schools, and the older pro- 
grams did not emphasize the scientific method or its applications. 

Summary. The IAEP results for both the mathematics and the science assessments were 
considered to be in line with the available committed resources. With the current 
implementation of new elementary and junior-high-school mathematics curricula, and 
with an increased focus on science instruction at these levels, we expect the results of all 
New Brunswick 13-year-olds to improve in the future. 
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Total 
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(K-12) 
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13-YearOJds 
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School 
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Number cf 
Days in 
School Year 


Average 
Class 
Size 


Provincial 
Curriculum 


9.001.000 


1.878.000 


102.000 


6 


185 


23.2 (elem,) 
23.0 (sec.) 


Ves 



Context 

Enrollments in publicly funded schools (public and Roman Catholic Separate) account 
for 97 percent of the school population for ages 6 to 16. Catholic schools educate about 
one-third of these students. Five percent of the population ages 6 to 15 attends 
French-language schools. These schools Jor the most part, are part of the Separate school 
system. The last decade has seen a significant increase in "new Canadians' (recent 
immigrants), and multiculturalism is an important component of government policy at 
both the provincial and federal levels. 

The Ministry of Education issues official curriculum guidelines and lists of approved 
textbooks. No formal streaming or tracking is provided through eighth grade, but 
students are expected to choose either the advanced (university^bound). general, or basic 
level for their ninth-grade courses. Students can obtain a secondary-school diploma 
following completion of 30 credits. Each credit involves 100 to 120 hours of instruction; 
16 credits are compulsory for most students. Students wishing to attend university must 
take six credits at the academic course level. Prior to 1968. the Ministry of Education 
administered province-wide examinations as a basis for awarding the grade 13 diploma. 
These examinations were replaced in 1968, for postsecondary admission purposes, by the 
Ontario Test for Admission to College and University. These tests were subsequently 
discontinued in 1974. 

Ontario participated in both the IEA Second international Mathematics Study and the 
IEASecond International Science Study during theearly 1980s. Interest in province-wide 
assess-- ,-nt grew as a result of these activities, and 1987 saw the initiation of the 
proving il review process. Through the use of school-based multiple matrix item sam- 
pling, provincial levels of student achievement are obtained in the areas of science, 
mathematics, and first language. Teacher and school information, including student 
opportunity-to-learn data, are also collected. A pilot review of Canadian Studies Geogra- 
phy at grades 9 and 10 was conducted in 1986-1987, and grades 1 1 and 12 advanced-level 
Chemistry and Physics followed in 1987-1988. During the 1988-1989 school year, 
provincial reviews of grade 6 Reading and Mathematics are being conducted. The review 
process operates on a five-year cycle, assessing two subject areas each year. 

ThelAEP involvement comes at a time of heightened interest in education. Two recent 
government-initiated reports were critical of achievement levels obtained by students, 
and a select committee of the legislature has been established to examine the goals and 
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directions in education. This study provides the first clear evidence of levels of Ontario 
Anglophone and FVancophone student achievement in relation to achievement in other 
Canadian provinces and other countries. 



Commentary 

Educational policy and practice is in a period of transition and renewal. The perfor- 
mance of students can best be understood within this context. 

Greater emphasis is being placed by the Ministry of Education on specificity of 
expectations for achievement in mathematics and science. New guidelines, which stress 
learning outcomes as well as process goals at all levels of the science curriculum, were 
issued in 1987 and 1988. A recent mathematics guideline at the intermediate level sets 
out detailed objectives. The present renewal of the primary and junior curricula includes 
the development of a policy document in mathematics. 

Until recently, Ontario has not had standardized accountability mechanisms such as 
provincial assessments or examinations. Ontario's participation in both the 1EA Second 
International Mat hematics Study and Second International Science Study helped initiate 
a new provincial review process. These provincial reviews are based on the curriculum 
and help meet demands for public accountability and provide evidence of curriculum 
effectiveness, including measures of student achievement. The provincial review program 
is also used to support curriculum renewal at the provincial and local level Continued 
participation in international studies will provide further comparative evidence. 

Financial and legal recognition of French-language schools has a short history in 
Ontario. There continues to be a shortage of French-language educational resources and 
the relatively small percentage (5%) and the wide dispersion throughout the province of 
the Francophone community contributes to the difficulty of meeting the educational 
commitment to French-language education. 

It is expected that Ontario's involvement in this international assessment will help to 
focus continuing efforts toward educational renewal in the province. 
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23.2 (elem,) 




6,541,000 


1,209,000 


74,000 


6 


180 


24.2 (sec.) 


Yes 
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Context 

Education is compulsory for all children from the beginning of the school year in 
which they reach the age of 6 to the end of the school year in which they reach the age 
of 15- Children can attend kindergarten if they are five years old and about 99 percent do 

so. 

The compulsory part of education is composed of an elementary level and a secondary 
level Elementary education usually covers six years of study, although a child may 
occasionally go on to secondary school after only five years. In any event, students must 
start the secondary level after seven years in an elementary school The duration of 
secondary-level studies is usually five years. Students enrolled in a vocational program 
may continue tor begin) their program during a sixth year. A school year comprises a 
minimum of 180 days of classes at both the elementary and secondary levels. At the 
preschool level, a school year represents 180 half-days. 

Local school boards are responsible for providing educational services at the elemen- 
tary and secondary levels in French, English, lnuit, or Amerindian languages according 
to the current regulations. Private education involves 3,7 percent of the students at the 
elementary level and 16.4 percent at 'he secondary level Funding from the province 
covers approximately 60 percent of the operating expenses of subsidized private institu- 
tions. 

The Ministry of Education establishes the programs of study that are offered in the 
schools. Each year the Ministry evaluates several of the programs being taught. This 
evaluation process involves administering examinations to a sample of students as well 
as sending questionnaires to teachers and administrators. The science an J mathematics 
programs for 13-year-oIds were recently evaluated using this process. 

Each subject in a student s program is evaluated in light of the course objectives. In 
order to obtain a secondary-school diploma a student must accumulate a minimum of 
130 credits, The student must also pass compulsory examinations, most of which are 
prepared by the school board* although some are prepared by the Provincial Ministry, 
The latter sets uniform examinations in some basic disciplines for students in the 
Secondary IV and V, The passing grade is 60 percent for all subjects at the secondary level 
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The probability of a student finishing secondary school and obtaining a diploma was 
63 percent in 1987, Students who wish to pursue postsecondary studies do so at the 
college level: in 1987, 61 percent of young adu T ts continued their cducat ; on in colleges. 



Commentary 

Almost all 13-year-olds are in either 3rade 7 or grade 8. These two grade levels 
constitute the First Cycle of secondary education in this province, 

Mathematics. The program of study m mathematics is compulsory and uniform for all 
students enrolled in the geaeral-education courses of study. An analysis of the tasrs 
presented in the IAEP mathematics assessment reveals that theya^e closely related to the 
mathematics programs that Quebec students follow in elementary and secondary 
schools. According to educators within the school community, the o erall results on this 
assessment are comparable to those obtained in previous evaluations and satisfy their 
expectations of students' performance. 

Science. Thirteen-year-olds have acquired most of their science understanding from 
three existing programs of study: Natural Science (elementary level). Ecology (grade 7), 
and Physical Sciences (grade 81 These programs are compulsory and uniform for all 
students registered in general education courses of study. 

Many IAEP questions focused on topics that are not addressed in the existing 
programs of study. This lack of any strong relationship between the IAEP assessment 
content and the science programs seems to have been corroborated by the responses of 
teachers in the opportunity-to-learn questionnaire. Because of this disparity between 
program content and assessment content, it is rather difficult to determine to what 
extent the Quebec student performance corresponds to educational expectations. To 
some extent, the Quebec assessment surveyed the scientific knowledge and awareness 
that students acquire from their cultural and social milieus, 

Finding. Overall results in the French and English populations were comparable. Ma!e 
results were slightly higher than female results in science but not mathematics. A high 
percentage of students stated that they like school "a lot" or +t a little." For many, 
mathematics is considered an important subject. Students reported that they iearn 
mathematics by listening to the teacher explain a lesson and then working on problems 
alone, rather than spending time working problems in small groups. 
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Tolal 
Populalion 


School 
(K-12) 
Populalion 


Number ol 
13-Year-0lds 


Age 
School 
Begins 


Number of 

Days in 
School Year 
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Size 


National 
Curriculum 


37.683,000 


7.571,000 


462,000 


6 


180 


30 


Yes 



Cents::! 

Twenty years ago, special concerns about education were raised in Spain. This fact 
coincided with the rapid economic growth that took place in the 1960s. During this 
process of economic development the need for a well-educated population became a high 
priority. Spain's middle class has satisfied its basic needs and is beginning to acquire 
economic capital that is available for investment, Education i$ viewed as a long-term 
investment, which can provide middle- and working-class children with an opportunity 
for social mobility, and as an important instrument in reducing social inequalities. 

For these reasons and the opening of Spain to the developed world, all of Spanish 
society has exhibited a renewed interest in the educational system. In particular, this 
concern was articulated in the General Education La>v of 1970. This law established the 
current organization of the educational system: preschool, ages 2 to 6; general basic 
education, ages 6 to 14 (compulsory and free); secondary education, ages 14 to 18 or 19. 
The secondary education level offers two possibilities: baccalaureate, ages 14 to 17, and 
course of university orientation, ages 17 to 18; and vocational training, ages 14 to 19, 

Gradually, the desire for universal education codified into law is being realized and, as 
time goes by* the percentage of children in school has reached 100 percent at the genera! 
basic education level Even though educational enrollment goals have been met, Spanish 
society is now convinced that attendance alone does not guarantee the desired equality 
of opportunity. Therefore, more than 15 years later, the need to reorganize the educa- 
tional system has resurfaced. This reorganization includes adjusting the system to fit the 
current environment and addressing the qualitative aspects of education. There is a 
strong political will within the current government to pass educational reform legislation 
during the 1989-1990 school year. 

Since the enactment of the Constitution of 1978. the central government has been 
responsible for all the functions and resources that are essential io assure the basic 
operation of the educational system: degrees, requirements for promotion, curriculum 
and scheduling standards, basic requirements for teachers, inspection, etc. But Spain is 
divided into 17 different autonomous communities, six of them self-governed with regard 
to education. These regions have the power to control the promotion of regional culture, 
some curriculum and schedule requirements (teaching of their own mother tongue in 
addition to Spanish, for example ), and supplemental educational budgets, apart from the 
national budget assigned to each community. Therefore, there exists a minimum curricu- 
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lum and schedule common to all students that can be added to hut never decreased in 
each of these six territories. This diversity must be accommodated as plans for organiza- 
tional and curriculum reform are considered and implemented. 



Commentary 



At present, Spain is involved in a process of major reform of the educational system. 
The reform process focuses on teaching methods in addition to school organization and 
curricular content. Currently, the general opinion is that education is very traditional, 
since the pupil is viewed as the recipient of knowledge and p!ays a role of passive 
participant in the process. Results of the IAEP tend to corroborate this impression. 
Students reported that most of their class time is spent listening to the teacher explain 
a lesson and that the help they get from the teacher is less than they receive from their 
classmates. 

On the other hand, the results do not confirm a current criticism that accuses teachers 
of emphasizing factual learning and spending less time on the applications of this 
knowledge. The data seem to show that in both mathematics and science curriculum, 
coverage is very balanced. Students obtained mean percents correct that ranged between 
about 50 and 70 percent in all the topics. In spite of this, the relative position of Spain 
with respect to the other countries and provinces is vory different, depending on the 
specific math or science topic. This leads to the conclusion that each country or province 
has its own set of priorities in each curriculum subject. 

In addition to the great similarity of results within the different topics, there were wide 
performance differences for individual questions. Researchers have suggested perform- 
ing factorial or cluster analyses of the data in order to study item groupings. 

Homework was not consistently related to achievement in the two subjects assessedn 
Students whose achievement was at the median did a lot of homework, got little help at 
home, and reported the most indifferent attitude toward school- 

Most populations participating in the assessment found differences in the performance 
of males and females in science but not in mathematics. In Spain, boys' achievement was 
better than girls' in both assessments. This fact confirms that there is still discriminatory 
treatment of girls in schools, at least from a qualitative point of view. In the formal 
educational levels, quantitative discrimination has disappeared; schooling rates are 
similar for boys and girls. In primary education (6 to 14 years old) the rate is 100 percent, 
and in secondary education (14 to 17 years old), around 60 percent for both sexes. 
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25.9 (elem.) 


Yes 


55,196,000 


9,221.000 


683.000 


5 


190 


21.0 (sec.) 


(as of 1 988) 



C onte?: ? 

During the past few years, there has been increasing concern about standards of 
education. The recent Education Reform Act (1988) proposes major changes in educa- 
tion, including the introduction of a national curriculum and the establishment of a 
national testing system. Mathematics and science will form part of the national core 
curriculum to be studied by alt pupils of compulsory school age. Students will be tested 
on Lhts curriculum at ages 7, 11. 14, and 16. The reports of the working groups on the 
national mathematics and science curricula were published in August 198G zr\d are 
expected to have a profound impact in the 1990s. 



Mathemalics. In 1982, the Cockcroft Committee recommended that mathematics teach- 
ing at al! levels should include opportunities for exposition by the teacher, discussion, 
practical work, practice of skills and routines, problem solving, and investigational work. 
Funds were provided by central and local government for curriculum development and 
assessment projects and for advisory teachers to work with schools, in order to promote 
a broader approach to mathematics teaching. The new 16+ examination, the General 
Certificate of Secondary Education, was administered for the first time in 1988 and has 
a more practical and problem-solving outlook than its predecessors, 

A source of particular concern is the impact of new technology. So far, schools have not 
taken much advantage of the availability of electronic calculators and computers. 
However, the increasingly widespread use of these machines in society has been recog- 
nized in the recommendations to develop attainment targets and programs of study for 
children ages 5 to 16. 

Science. Science education in the United Kingdom tends to be more process oriented 
than in m^ny other countries, and some schools already offer their students a wide range 
of experierces in which to develop their scientific skills. Nevertheless, science educators 
have expressed concern about the shortcomings of British science education. At the 
primary level, teachers often lack a working knowledge of the sciences, and at the 
secondary level, there has been concern that syllabi were overloaded, emphasized recall 
rather than understanding, and reflected the traditional disciplines of Physics. Chem- 
istry, and Biology at the expense of other areas such as Astronomy, Earth Science, and 
Technology. Furthermore, the governments policy that science should be studied byal! 



students of compulsory school age, whether or not they arc likely to follow a career in 
science and technology, has not always been fulfilled, 

Pressures for change have increased, partly because of the needs of industry for a 
scientifically trained work force, and partly as a result of the government's policy to raise 
standards in all areas of the curriculum. 



Commentary 

Mathematics. In comparison with oth^r populations, the United Kingdom did quite well 
ranking first, second, or third in most topics. In Measurement and Numbers and Opera- 
tions, the picture was more bleak, with the ranks being ninth and elevenths respectively. 
For some questions, the success rate of the British students was considerably lower than 
that of alt other countries. Where it is possible to compare 1AEP items with similar ones 
used in British surveys {mainly APU), the results seem comparable, with some a little 
below the British results and others a tittle above, However, students^ poor performance 
in Numbers and Operations is put into perspective by that of other countries in this 
survey. 

Gender differences in mathematics can only be considered impressionistically. It 
appears that, as in the 1931 1EA Survey, the differences are not as marked in measurement 
items as would be expected from APU results at age 11 and 15- It seems possible that 
there may be a leveling off of the difference or even a redaction in the differences between 
boys and girls aged 11 to 13 years. 

Science, The United Kingdom ranked third in Physics and Earth and Space Sciences, 
fourth in Nature of Science, and sixth in the Life Sciences and Chemistry. Britain's 
relatively high ranking on the Earth and Space Sciences is a little surprising since this 
topic is not normally emphasized in science classes. It may be that the information 
required to answer the questions was obtained from other subject disciplines or non- 
school sources such as the media. Se^eof thegreatest gender difference in favor of hoys 
were found in items concerned with Earth and Space Sciences. 

For many years there has been widespread concern about the small proportions of girk 
continuing to study the Physical Sciences after age 14, and the recent report of the 
national science curriculum working group has recommended that further research 
should be undertaken into the different perceptions of and reactions to science by boys 
and girls. The results of this study show an overall pattern (which held for most 
populations and most science topics in the assessments) in which boys achieved higher 
scores than girls, However, the United Kingdom was one of two populations with no 
significant gender differences (the other was the United States), In addition, girls in the 
United Kingdom scored better than boys on questions concerned with the Nature of 
Science, In the recent 1EA study (1988), for which testing took place in 1984, the size of 
the gender difference among 14-year-olds in England was slightly above the average for 
the 17 countries compared. However, it was slightly smaller than that of the United 
States, Canada (English-speaking), and Korea. 
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241,000,000 


45,900,000 
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5 
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The interest in improving standards in mathematics and science education has been very 
high since 1983, fueled by the two recent IEA studies in which United States perfor- 
mance compared unfavorably with that of other countries. Education reform was an 
important issue in the recent presidential election. The last period of great interest in 
mathematics and science instruction occurred in the 1960s after Sputnik. 

During the past 20 years American education generally seems to have succeeded in 
strengthening the basic skiils of allo( its students — minority and economically disad- 
vantaged children as weli as the sons and daughters of more affluent parents. National 
assessments consistently reveal a weakness in higher-order thinking skills in all subjects, 
and there is a growing concern and determination to improve these skills. Policymakers, 
business leaders* and educators agree that young Americans will need these skills in the 
21st century if they are to lead satisfying lives and if the national economy is to prosper. 
Newspapers and other media discuss these issues regularly. 

AN young people are required by law to attend school until they reach the age of 16. 
Most begin school at about age 5 and complete their secondary education at about age 
17 or 18. About 15 percent drop out before completing the high-school program. There 
are no nationally mandated curricula in mathematics or science- but some of the 50 states 
publish recommended courses of study for these subjects. Mathematics i*s taught to all 
students every day during the first eight years of school, usually by the regular classroom 
teacher, and is a!so required in most secondary schools. Less than half of all high-school 
students take courses beyond first-year Algebra and Geometry. The National Council of 
Teachers of Mathematics is recommending new standards that reflect a national consen- 
sus for mathematics education- 
Science is much less formally taught in elementary schools- with usually about two to 
three hoursof instruction per week. In secondary school 85 percent of the students study 
Biology for a year or more, but only about 35 percent take Chemistry and fewer than 10 
percent elect to study Physics. 

Educators who have analysed student weaknesses tend to fault the curricula, lack of 
laboratory facilities- and the inadequate training of teachers. There are strong initiatives 
underway to strengthen curriculum and to increase the competence teachers. More 
mathematics and science courses are being required of all students. The content of those 
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courses is being made more rigorous and tests are being required to monitor progress. 
The challenging priorities of American education include an equal and quality education 
for all American children and high levels of achievement in all curriculum subjects. 



Commentary 

The results of the IAEP confirm the findings of other international and national 
research projects on mathematics and science achievement. Respected educational 
leaders have stated publicly that achievement levels are distressingly low and unaccept- 
able, especially in view of the requirements of today's and tomorrows technological 
environments. 

Mathematics. The position of the United States as last in overall achievement heightens 
concern for the future in an increasingly competitive world. Todays 13-year-olds will be 
the voters of 1993 and the country's employees of 1995, Their ability to understate 
mathematical concepts and solve problems will determine their individual success and 
the collective prosperity of the nation. While it is satisfying to see that close to 100 
percent of our students from all segments of our society with a mastery of the basics 
(Level 300), the fact that only 40 percent of them are able to solve two-step problems 
(Level 500) is a matter for grave concern. Percentages at Levels 600 and 700 are even 
more modest and suggest that the pool of trained talent from which to draw our future 
scientists, engineers, and technicians is smali indeed. 

Comparisons with the more successful competitors suggest examining the impact of 
heavy television watching by students and the small amount of mathematics homework 
typically done. The importance attached to mathematics by schools and society in general 
also may be a factor. 

Science. The United States has traditionally thought of itself as technologically innova- 
tive and in the forefront in science. These results are sobering and pose a serious 
challenge to our position in the world community. It is satisfying to observe that almost 
all of our 13*year-olds, including those from the most economically disadvantaged sectors 
of our society, know everyday science facts (Level 300). Nevertheless, that only 42 percent 
of them can use scientific procedures and analyze scientific data (Level 500) is clearly 
unacceptable. Students attitudes about the subject need to be strengthened, and the 
subject must become a more important part of the school curriculum. There arc major 
efforts underway to redefine ancJ strengthen the treatment of the subject, to train 
teachers, and to enlist parents and businesses support for the study of science. These 
efforts must be given high priority. 

Summary, The findings of this study will present yet another opportunity to call the 
attention of policymakers, the business community, and America s parents to the poten* 
tia! problems the country will face if the mathematics and science curricula are not 
strengthened. Past successes by the schools, e,g. P with basic-skills improvement, offer 
convincing evidence that if there is clear and common agreement r>n our goals, the 
educational system will be responsive. 
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Summary and CencSusaons 



The similarities reflected in these pages are probably as remarkable as the deferences. 
Countries as old as Korea and as young as Canada can boast that ^5 percent of their 
13-year-olds have mastered the basic arithmetic operations and can solve simple prob- 
lems in mathematics. Populations as diverse as those of British Columbia and Spain have 
taught all of their Kl-year-olds the basic scientific facts measured by thi* assessment. 

Viewed historically, these are major achievements. These statements support the hypoth- 
esis that schools all over the world share common goals and similar curricula- Vet what 
captures attention are the differences, Thirty three percent of Korea s 13-year-olds can 
apply intermediate scientific knowledge and principles in designing experiments and 
interpreting data. Less than 10 percent of their Irish counterparts are able to do as well. 
Forty percent of these same Korean young people are able to apply a range of strategies 
to solving fairly complex mathematics problems. Less than 10 percent of their United 
States age-mates can do so. 

Reactions to these kinds of findings elicit competitive emotions and questions con- 
cerning why these differences exist. 

□ Are certain schools "better"? 

□ Is motivation different? 

□ Do students work harder in some countries? 

□ Do certain societies attach greater importance to science or mathematics? 

These assessment data, along with the comments of the representatives of the various 
countries and provinces, credit the disparities to a combination of factors. Certain facts 
&eem clear: 

□ Many societies are currently focusing attention on education and curricu- 
lum reform, This kind of attention seems to make a difference. 

□ Attitudes of students ar I parents are important, 

D There is little consistency in the relationship between types r>f classroom 
activities and achievement. 

□ Students learn a lot about mathematics and science outside of the 
classroom. 

The debate concerning the importance of these two school subjects during the next 
few decades will continue. The issues have to do with economic well-being, international 
leadership, and national defense- Where will the world's innovative discoveries, new 
solutions, and creative products come from in the future? Does it matter? 

What seems reasonable to assume is that if any country's young people are well 
educated, they will have increased opportunities for more productive and more satisfying 
lives. 

The hope is that these kinds of studies will encourage and illuminate the discussions 
that will clarify opinions, sharpen objectives, and improve educational effectiveness. 
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J3ackr> round 

Since 1983 Educational Testing Service 
(ETS) has administered the National Assess- 
mentof Educational Progress <NAEP) as well 
as related projects, such as state assessments 
and the International Assessment of Educa- 
tional Progress reported here. NAEP is an 
ongoing, congressionally mandated project 
established to conduct national surveys of the 
educational attainments of students in the 
United States. Its primary goal is to determine 
and report the status of and trends over time 
in educational achievement NAEP was ini- 
tiated in 1969 to obtain comprehensive and 
dependable national educational-achieve- 
ment data in a uniform, scientific manner. 

After conversations with the representa- 
tives of several foreign countries, ETS staff 
developed a proposal for an International 
Assessment of Educational Progress (IAEP) 
designed to achieve two objectives: 

□ Explore the feasibility of reducing the 
time and money requirements for 
international comparative studies by 
capitalizing on the NAEP materials 
and procedures. 

□ Permit interested countries to experi- 
ment with NAEP technologies to 
determine their appropriateness for 
local evaluation projects. 

In r ebruary 1987, ETS staff called a meet- 
ing of interested parties to discuss the feasibil- 
ity of an international assessment project The 
results of that planningsession were a series 
of requirements and a timeframe foreonduct- 
ingan assessment of mathematics and science 
achievement of 13-year-olds. 

With these specifications in place. ETS was 
able to obtain funding from the Department 
of Education and the National Science Foun- 



dation for overall coordination, sampling, data 
analysis, and reporting. Participating coun- 
tries and provinces acquired support for local 
data collection and coordination. 

Project implementation was carried out 
through a series of meetings in 1987 and 1988 
devoted to selecting assessment items, review- 
ing pilot-test results, and reviewing and inter- 
preting final results. Decisions were made col- 
laboratively, and follow-up coordination was 
provided by ETS staff. 

Student Assessment Instruments 

Assessment questions were selected from 
the pool of 281 mathematics and 188 science 
questions used in the 1986 NAEP. Mathemat- 
ics and science experts from each country' and 
province reviewed questions in terms of how 
well they assessed aspects of their country's 
curricula and how well they could be trans- 
lated and/or adapted to reflect the local cul- 
ture. Participants selected 90 mathematics 
items and 83 science items for pilot testing. 
The questions were selected so that a variety 
of content categories and skill levels would be 
represented in each subject (see chapters 3 
and 6 for descriptions of topics). The final 
selection also reflected a range of difficulties 
and item characteristics. 

Selected questions were translated from 
English to French. Kcrtan. and Spanish and 
then independently translated from the non- 
English language back to English. The back- 
translated versions were compared with the 
originalEnglish to ensure that the transla- 
tions were accurate. Questions were also 
adapted for cultural differences. For example, 
units of measurement, the names of children, 
and species of plants and animals were 
changed to reflect local usage and environ- 
ments. In addition to the translated assess- 
ments, two additional English forms were 



developed to adapt to local usage and en ;iron- 
merits one for use in Canada and one for use 
in the United Kingdom and Ireland. Questions 
were then pilot tested with at least 100 stu- 
dents in each location. Participants assessing 
students in two languages conducted pilots 
for each population. 

The results of the pilot tests were used by 
the participants to select the final 63 items in 
mathematics and 60 items in science that met 
the targets for content categories and skill 
levels identified earlier. All of the science 
questions used a multiple-choice format. 
Fourteen of the mathematics questions were 
open-ended and required students to calcu- 
late and write their answers in their booklets. 
Translations were improved as needed and the 
final versions were back-translated and 
checked for accuracy at ETS. 

Two assessment booklets were assembled, 
one for each subject. Booklets contained 
three sections* each with about 20 questions 
ordered from easy to hard. Students were 
allowed 15 minutes to finish each section. In 
the United States and some other locations, 
six mathematics items and six science items 
drawn from the IEA assessments were added 
tothelAEP assessment, two at the end of each 
section. Results of these items are not 
included in this report. 

In addition to the cognitive assessment, stu- 
dents were asked 32 background questions 
about their mathematics and science instruc- 
tion, their attitudes toward these subjects, 
and related activities at home (television 
watching, homework, home involvement in 
science activities)* Most of these questions 
were selected from prior NAEP assessments, 
but in some cases they were developed specif* 
ically for the 1AEP. In addition to the core set 
of background questions administered to all 
participants, some countries and provinces 
added items of their own. 

Sample Design and Survey Response 

The four Canadian provinces decided to 
assess more than one population of students. 
New Brunswick, Ontario* and Quebec selected 
separate samples of English- and French- 
speaking students. British Columbia selected 



separate samples of public- and private-school 
students (all English-speaking), which were 
combined for d*ta analysis. The United King- 
dom selected its sample from England, Scot- 
land, and Wales (English-speaking students). 
The sampling frame did not include students 
from the inner London Educational Authority 
(approximately three percent of the total pop- 
ulation). Spain sampled students throughout 
the country and provided assessment booklets 
in Castilian, Catalan, Basque* Galacian. and 
Vaiencian languages. All but about 250 of the 
students in Spain took the assessment in 
Castilian. and only these students were 
included in the data analysis. 

The sample designs for each of the popula- 
tions may be described as stratified cluster 
samples. The participants, however, were free 
to design their surveys independently as long 
as certain specific rules were followed. The 
principal requirement was that their data be 
amenable to analysis as a paired cluster 
design, thus permitting the use of a jackknife 
procedure for the estimation of standard 
errors. 

With two exceptions, all of the surveys fol- 
lowed the same two-stage sampling process, 
In the first stage, schools were selected with 
probabilities proportional to estimated size 
(number ofl3-year-old students). At the sec- 
ond stage, subsamples of students were ran- 
domly drawn from within each selected 
school. Typically, about 100 schools were 
selected at the first stage and ahout 2.000 
students at the second stage. 

In contrast to the other surveys in which 
individual students were sampled within each 
.selected school. New Brunswick (French) 
selected intact classes of students for assess- 
ment. In the United States, the 1AEP booklets 
were included along with the H)38 NAEP. 
which used a three-stage design in which the 
first-stage selections were metropolitan areas 
and counties, the second-stage units were 
schools, and students constituted the third. 

In 1988. NAEP booklets were administered 
to two equivalent half samples of students, 
each including approximately ].000 students 
meeting the age definition. The first sample 
was assessed from January 1988 through mid- 
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March and the second, from mid-March 
through May. The results for the United 
States are baaed on the responses of siudems 
in the first half sample. These students were 
sampled from about 200 schools, thus mini- 
mizing the effects of clustering. 



Brunswick (Trench) was lower than for other 
populations. 73 percent as compared with 
percent or higher. This low figure may he due 
to inaccurate records for the number of eligi- 
ble students. 



Details of selection procedures and thecal- 
culation of student weights for each separate 
assessment survey are provided in the 1AEP 
Technical Report. Achieved sample sizes and 
cooperation rates are provided in TABLES A 1 
and A.2. 

The school participation rate across all 
assessment materials for the United Kingdom 
was 70 percent, compared to 89 percent or 
higher in other locations. In New Brunswick 
(English), school participation rates for the 
background questionnaire were SO percent, 
because one large school board had a long- 
standing policy on surveys that did riot permit 
its students to provide background informa- 
tion (grade, sex. and other variables). The 
average student participation rate for New 



Data Collection znd Scoring 

All countries and provinces used standard- 
ized administration procedures. School per- 
sonnel or external administrators followed 
the iame administration script. Each cogni- 
tive mathematics and science assessment 
required 45 minutes to administer, and the 
time devoted to the background questions 
varied depending on the number of additional 
items added by participants. 

In all locations except the United States, 
students completed one assessment booklet 
in the morning and the other assessment 
booklet in the afternoon. In a random half of 
the sampled schools, mathematics was 



|f Number of Students Assessed 



IAEI» # 3S 





Number of Students 
Assessed (Raw Data) 




Weighted Number 
of Students Assessed 




Mathematics 


Science 


Background 




Mathematics 


Science 


Background 


British Columbia 


3,025 


3,025 


3,025 




33,589 


33,589 


33,590 


Ireland 


2,253 


2,244 


2,257 




65,927 


65,927 


65.928 


Korea 


2,243 


2.243 


2,243 




902,516 


902,516 


902,516 


New Brunswick (English 1 


2,047 


2.041 


2,402 




6,175 


6,159 


6.446 


New Brunswick (French; 


1,548 


1.539 


1,520 




3,536 


3.539 


3,500 


Ontario (English) 


2,003 


2.018 


2,015 




97,154 


97.605 


97,605 


Ontario (French) 


2,075 


2.075 


2,075 




5.010 


5.010 


5,0^0 


Quebec (Fnglish) 


2,090 


2,013 


2,152 




5,934 


6,005 


6,053 


Quebec (French) 


2,186 


2,169 


2,189 




67,582 


67.433 


67,466 


Spain 


1,756 


1,756 


1,756 




462,158 


462,158 


462,159 


United Kingdom 


2,202 


2,202 


2,202 




682,999 


682.999 


683.000 


I'nited Slates 
^Mathematics sample) 


905 




905 




3,051,017 




3,051,017 


United States 
(Science sample) 




859 


859 






2,832,291 


2,832,291 
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assessed first followed by science, and in the 
other half the order was reversed. Typically, 
the background questions were administered 
after one of the cognitive assessments. In the 
United States, two equivalent samples of stu- 
dents were assessed, one for mathematics and 
the other for science. 

The assessment was administered in Febru- 
ary 1988, except in the United States, where 
the data were collected during the January 
through middle-March NAEP assessment. 

Assessment booklets were returned to a 
central location within each country and 
province. The shipments were checked for 
completeness, open-end mathematics items 
were scored as correct or incorrect following 
standardized scoring guides, and responses 
were keyed or scanned. 

Each country and province was responsible 
for developing a data file following a standard 
format, checking ranges of responses, and 
resolving inconsistencies in the data. These 
files were then sent to staff at the Quebec 
Ministry of Education where they were again 



checked. Each participant also sent to Quebec 
a random sample of booklets so that the data 
files could be checked against the original 
documents. Files were also sent to ETS at 
Princeton, where weights were calculated and 
added to the files for the United States and 
Canadian participants. Weights were verified 
for the other participants. 

Kean percepts Correct Analytic 

Data analysis was conducted by a research 
team at Laval University, Quebec, in consulta- 
tion with researchers and data analysts at 
ETS, Princeton. The first stage of analysis 
involved the calculation of the percentage of 
correct answers and standard errors for indi- 
vidual Questions and groups of questions. For 
each population, the weighted percentage of 
correct answers w;is calculated for each ques- 
tion. Students who omitted questions at the 
ends of sections because they did not reach 
them were excluded from the calculations for 
those questions. For each percent correct, an 
estimate of its standard error w: . calculated 
using a jackknife procedure. P. ^centages and 
standard errors we.*e calculiA^ for all stu- 



VVTA9LEA.2 

*S& C r» H ifi n 1 i si h n A Q 1 1 1 


mm* 


i hp 1*3 tin 








m 
StiS 




:&jl„q u ii u u J ■ d J i u , o i u 


U Cjll;j^L>U- 


JUCIdUU 


U Tidico 
























Schools 




Students within Schools 
(Mean Hale) 




Mathematics 


Science 


Background 


^ Malfi&matics 


Science 


Background 


British Columbia (Private) 


1.000 


1,000 


1.000 




0.977 


0.977 


0.977 


British Columbia (Public) 


1.000 


1.000 


1.000 




0.960 


0.960 


0.960 


Ireland 


0.971 


0,971 


0.971 




0.901 


0.398 


0,903 


Korea 


0.943 


0,943 


0.943 




0.975 


0.975 


0.975 


Mew Brunswick (English) 


0.954 


0.954 


0.800 




0.924 


0.926 


0,927 


Mew Brunswick (French) 


0.903 


0.906 


0.896 




0.735 


0.729 


0729 


Ontario (English) 


0.963 


0.963 


0.963 




0.938 


0.938 


0,937 


Qntario (French) 


0,972 


0.972 


0.972 




0.956 


0 955 


0,956 


Quebec (English) 


0.968 


0.968 


0.968 




0.981 


0,960 


0.981 


Quebec (French) 


0.947 


0.947 


0.947 




0.974 


0.969 


0.977 


Spain 


0,893 


0.893 


0,893 




0,976 


0.976 


0.976 


United Kingdom 


0702 


0.702 


0,702 




0.943 


0.943 


0.929 


United States 


0,369 


0.872 


0,872 




0,899 


0 897 


0.901 
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dents in each population and also by sex and 
grade, 

Percentages were also averaged across 
groups of items for each population, Each 
assessment was divided into several topics 
that experts agreed reflected content areas or 
skills in mathematics and science, Average 
percents correct and standard errors were 
computed across the questions in each of 
these topics and across all questions in mathe- 
matics and all questions in science, Items that 
did not perform in a similar fashion across all 
populations were excluded from these analy- 
ses (see discussion of differential item func- 
tioning), 

IRT Scaling 

The second stage of analysis involved the 
scaling of mathematics and science results 
using item response theory (IRT) technology, 
Two scales were developed, one characteriz- 
ing mathematics performance, the other sci- 
ence performance. The underlying principle 

of this methodology is that when a number of 
items require similar skills, the regularities 
observed across patterns of responses can 
often be used to characterize both the respon- 
dents and the tasks in terms of a relatively 
small number of variables- When aggregated 
through appropriate mathematical formulas, 
these variables cap! re the dominant features 
of the data- Using the scale, it becomes possi- 
ble to talk about distributions of proficiency 
in a population or subpopulation, and to esti- 
mate the relationships between proficiency 
and background variables, 

IRT defines the probability of answering a 
given question correctly as a mathematical 
function of proficiency or skill and certain 
characteristics of the question. Specifically, 
the IAEP used a three-parameter logistic 
model- 

Deferential Item Functioning 

Because the 1AEP was administered to 12 
populations in six countries and in four lan- 
guages, additional steps were added to the 
scaling procedure to ensure that scales were 
summarizing the same constructs in all 
groups, First, item parameters were estimated 



for each population separately £^d results 
were examined to ensure that the responses 
fit the IRT model, No questions were rejected 
for lack uf model lit for any population at this 
stage, 

Secondr the Mantei-Haen:zel statistic was 
used to identify questions that exhibited dif- 
ferential item functioning (D1F), A question 
functions differentially across populations if 
students of equal ability but from different 
populations have different probabilities of 
answering it correctly, For these analyses, the 
students from the United States were identi- 
fied as the reference group and students from 
each of the other populations were identified 
as focal groups. The DIF analyses were per- 
formed separately for each population outside 
of the United States and compared with the 
United States for each mathematics and sci- 
ence question. The DIF statistic for a given 
focal group and Question estimates the differ- 
ence in the proportion of correct responses to 
the question between members of the focal 
group and members of the reference group 
after the students within the two groups have 
been matched on their overall ability level- 

The ] 1 DIF statistics fone for each popula- 
tion outside of the United States) for a given 
question were then standardized by dividing 
by their standard errors. A measure of the 
degree of DIF across all populations was then 
computed as the corrected sum of squares of 
the II standardized DIF statistics. The ques- 
tionswere ranked in terms of their across-pop- 
ulation DIP statistic and the magn of 
their ordered DIF statistics was compared 
with reference values that would be expected 
to be obtained if th p re were no differential 
item functioning for any question, Questions 
whose across-population DIF statistics were 
significantly larger than the reference values 
were identified as outliers. These questions 
were deemed as exhibiting differential item 
functioning. In mathematics, one question 
was identified as inappropriate for scaling 
purposes, and in science six questions were so 
identified, 

Estimation of Proficiency levels 

Given these two steps for ensuring that 
responses of students from all populations 
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met the requirements for IRT scaling- two 
scales were developed that summarized per- 
fprmance across all populations, one for math- 
ematics and one for science. To equally repre- 
sent all populations in the scaling process, 
random samples of 400 students for each sub- 
ject were selected from each population- and 
the item parameters for the subject were esti- 
mated on the combined sample of 4,800 stu- 
dents. These parameters and the patterns of 
student responses were used to estimate abil- 
ity levels for each assessed student. Results 
were arbitrarily expressed on a hypothetical 
scale that ranges from 0 to 1,000 and has a 
weighted mean of 500 and a standard devia- 
tion of 100 across all populations, Although 
the results for mathematics and science are 
expressed in the same units, it is not appropri- 
ate to compare scores on one scale with scores 
on the other. 



Scale An charing 

One of the lAEPs goals is to describe what 
students know and can do in easily under- 
stood terms and to stimulate debate about 
whether those levels of performance are satis- 
factory. An additional benefit of IRT method- 
ology is that it provides for criterion -refer- 
enced interpretation of levels on a continuum 
of proficiency. Although the proficiency 
scales range from 0 to 1,000. hypothetical^ 
more than 99 percent of the students* scores 
fall within the range of 200 to 500. The five 
levels chosen for describing results in the 
report are 300, 400. 500. 600, and 700. Each 
level is defined by describing the types of 
mathematics or science tasks that most stu- 
dents attaining that proficiency level are able 
to perform successful! >: each _s illustrated by 
a typical benchmark question (see chapters 1 
and 4). Data are provided that give the esti- 
mated proportion of students from each popu- 
lation that perform at or above each of the five 
proficiency levels. 

In the scale-anchoring process, the 1AEP 
identified questions from the 1988 assess- 
ment that were good discriminators between 
proficiency levels. Aquestlon w?s identified as 
a benchmark question at a given level if stu- 
dents at that level had at least a 65 to §0 
percent probability of getting the question 



right, while students a t the next lower level 
had a much lower probability of success: i.e. t 
less than 50 percent and at least 20 percentage 
points lower than the probability at the 
higher level. Mathematics and science experts 
examined these empirically selected ques- 
tions as well as the other questions in the 
assessment and used professional judgment 
to characterize each proficiency level. In some 
cases, only one or two Questions were identi- 
fied for a level (particularly for Levels 300 and 
700)- and experts had to ma ke inferences 
about prerequisite skills or transfer of skills 
that students might a Lo be demonstrating. 
They were aided by the previous experience of 
anchoring the same items when they were 
used iri the 1986 NAEP mathematics and sci- 
ence assessments. 



Opportun:£y-to-Lenm Ratings 

The assessment collected information from 
teachers on students* exposure to the material 
covered in the mathematics and science 
assessments. The purpose of collecting the 
opportunity-to-learn ratings was to see to 
what extent students in the participating pop- 
ulations had been exposed to various mathe- 
matics and science content areas. It was 
assumed that lack of coverage of a content 
area included in the assessment might be a 
reason for low performance in that area. 

In each participating school, a mathematics 
teacher or coordinator was asked to indicate 
the percentages of the seventh- and eighth- 
grade students in the school who had already 
had an opportunity to learn — anywhere in 
the school program — the concepts tested by 
each item in the assessment. (In the United 
Kingdom ratings were obtained for eighth- 
arid ninth-year students, the years in school 
that enroll the majority of 13-year-olds.) In 
some cases, all teachers within a grade in a 
school developed a consensus rating for that 
grade, and in some cases, several teachers in a 
grade provided separate ratings. Response 
options for the ratings were; ,+ all or most 
fmore than 75 percent*," "some (25 to 75 
percent)/' "few {fewer than 25 percent)." and 
"none." The same information was collected 
from science teacherr, and coordinators about 
the science questions. 
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The analyses of these data focused on 
schools in which more than 75 percent of the 
students had already had an opportunity to 
learn the content measured by a question. It 
was assumed that students in these schools 
had been exposed to the content. 



!n order to compare the opportunity- to- 
learn information with achievement results, 
the "most" ratings for grade 7 and for grade 8 
were weighted in proportion to the number of 
13-year-olds in each of these grades. Specifi- 
cally- for each school for each question, a 
"most" rating was assigned a value of 1 and all 
other ratings a value of 0. The value for the 
grade 7 rating was multiplied by the number 
of seventh graders and the school weight and 
the value for the grade 8 rating was multiplied 
by the number of eighth graders and the 
school weight The results were then added 
together. In the case of multiple ratings from 
the same school for a grade, ratings were also 
weighted proportionally, e.g.. each of three 
ratings was assigned a weight of one-third. 



Results were then summed across schools and 
divided by the Sum of weights to obtain an 
? -erage percentage of h+ most ++ responses for 
inch item. Appropriate standard errors were 
calculated using a jackknife procedure. The 
weighted percentages of +< most ++ responses 
were then averaged across Questions within 
each topic and standard errors of these statis- 
tics were calculated- 
Response rates for opportunity-to-learn 
ratings were low for several populations. Tor 
mathematics, the nonresponse rate of teach- 
ers translated into missing data for more than 
10 percent of the student population in New 
Brunswick (English}. New Brunswick 
(French). Ontario (English), Ontario 
(French), Quebec (English), and the United 
States. For science, the nonresponse rate of 
teachers translated into missing data for more 
than ten percent of the student Populations in 
Ireland. New Brunswick (English)- New 
Brunswick (French). Ontario (English). 
Ontario (French), Quebec (English)- Quebec 
(French). and the United States. 
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DATA 

' APPENDIX 

Mathematics Proficiency Means and Jackknifed Standard Errors 





TOTAL 


MALE 


FEMALE 


BRITISH COLfIMEIA 


539 B r? ?J 


5396 I28> 




IRELAND 


5043(3 7) 


SOS ? (6 ?) 


4995 <3 4) 


KOREA 


5676(27) 


S767 (34> 


5580 (3 9> 


NEW BRUNSWICK 1 ENGLISH) 


5290 (2.6) 


5256(4 3) 


5290 (3 4| 


NEW BRUNSWICK (FRENCH ) 


514 2 f33j 


si66(S2f 


5C4(36f 


ONTARIO (ENGLISH) 


Si6l (ai) 


5i?8(44f 


Sl«6(33r 


ONTARIO! FRENCH) 


431 Sl2 7) 


4605(3 4) 


4a?''30> 


QUEBEC LcHGuSh) 




534 1(30) 


5373(23. 


Quebec (FRENCH* 


&43Q(3 i) 


5453 (4 2) 


S393(30> 


SPAIN 


5H7(46j 


5232(53) 


4399(50f 


UNITED KINGDOM 


5099 (3SJ 


SO? CIS Ok 


Si25(39f 


UNITED STATES 


4739(3 5) 


4746(64) 


4732 (5 ik 



AMOUNTS OF WEEKLY MATHEMATICS HOMEWORK AMOUNTS OF DAILY TELEVISION VIEWING 





LESS THAN 


1.2 


3 HOURS 




0.2 


3-1 


5 HOURS 




1 HOUR 


H&URS 


OR MORE 




HOURS 


HOURS 


OR MORE 


BRITISH COLUMBIA 


53*. 3 (34) 


5453 (32) 


5*26 C ' O.i 


BRITISH COLUMBIA 


556 1 il 7) 


535 & 129) 


5094(3 7) 


IRELAND 


4972(39) 


5113(5 3) 


5320 re 6.i 


IRELAND 


5234(33) 


SOD 3 (35) 


4563(361 


KOREA 


5521 (4 9; 


56? 4 (35) 


6029 (4(n 


KOREA 


ssoorssi 


5606(261 


53? 3 jam 


NEW BRUNSWICK (ENGLISH) 


5222(4 3) 


5360 [4 4] 


5502 (*3) 


NEW BRUNSWICK 1 ENGLISH 1 


S48 7 (4 0) 


524 6(34) 


507 9 (7 i) 


NEW BRUNSWICK (FRENCH ) 


sisi co) 


ST? (4.9) 


&isa '4 2> 


NEW BRUNSWICK (FRENCH) 


5192(47) 


5163(36) 


5053 (4 m 


ONTARIO (ENGLISH) 


5i59(39) 


5114 (4 7) 


525 7 (41} 


ONTARIO (ENGLISH) 


537 0(4 &) 


515 4 (3&) 


465 9 I3.2> 


ONTARIO (FRENCH) 


*8Q0(33f 


474 3(33) 


5023 (4?] 


ONTARIO (FRENCH) 


497 4 (4 4) 


431 3(31) 


457 6(4 4) 


QUEBEC (ENGLISH) 


5325(30) 


5339130) 


S;4 7 (4 1) 


OUEBEC (ENGLISH) 


5434(30) 


534 3 (29) 


5136(33) 


QUEBEC FRENCH! 


S3&4(36) 


039.1 (4 31 


5597 (4 0i 


OUEBEC (FRENCH} 


5493(33) 


5402(36) 


5261 (511 




499 7 (6 7) 


5058 (52) 


533 9 (4 3) 


SPAIN 


5239(05) 


507 5(4 1) 


4306 (5S) 


united kingdom 


491 3 {4 6) 


5338 (35) 


6358 (73) 


united Kingdom 


529 1 (4 5) 


5201 (39) 


47S1 (46) 


united States 


4673 <S ?i 


4763(91) 


5066(109) 


UNITED STATES 


S94 018 2) 


t.%2 7 (59) 


442$ (7 3) 



Percentage of Students with Mathematics Proficiency 
At or Above Each Level and Jackknifed Standard Errors 





LEVEL 300 


LEVEL 400 


LEVEL 500 


LEVEL 600 


LEVEL 703 


BRITISH COLUMBIA 


997 10 1) 


9<-9(06) 


69.5 (i 2} 


233 (1 1) 


20 


(03) 


IRELAND 


933(06) 


B63(09) 


54? \\7\ 


"2(i0) 


06 


(02J 


KOREA 


996(01) 


953(051 


73 1 (1 1) 


396(13} 


4.9 


(051 


NEW BRUNSWICK (ENGLISH) 


999(01) 


955(06) 


654(14) 




■to 


CO 31 


NEW BRUNSWICK (FRENCH} 


99 6 ( 02) 


946 (0.7) 


583(i9> 


11.6 (121 


04 


(02l 


ONTARIO (ENGL&SHJ 


992(02) 


91 3 (0 7) 


533(1 5} 


T60 (1 3) 


1 4 


104) 


OKTARlO (FRENCH) 


93.7(03) 


648(1 11 


405(1 5) 


70 (0 7) 


01 (< 1) 


QUEBEC (ENGLISH) 


99 S C0 1 k 


96 7 10.4) 


67 3 ft 2] 


20 2 (1 0) 


14 


(021 


QUEBEC (FRENCH) 


999(01) 


97 2(0.4} 


727(1 3) 


21 6(1 3) 


\ 7 


(03) 


SPAIN 


992(02) 


90 Ml OJ 


570(2*) 


14 3 (18} 


1 3 


CO 4) 


UNITED KINGDOM 


935(0 3) 


667 (1 0) 


555(1.7) 


16.4 (t2l 


25 


(04) 


UNITED STATES 


966(0?) 


77 7 (21) 


40.3 (22t 


92 (TO) 


07 


(04) 



Mathematics Mean Percents Correct and Jackknifed Standard Errors by Topic 





NUMBERS 


RELATIONS, 








DATA 


LOGIC AND 




AND 


FUNCTIONS, 






MEASURE- 


GRGANL. 


RRC&LEM 




OPERATIONS 


ETC. 


GEOMETRY 




MENT 


zation t no. 


SOLVING 


British Columbia 


766(06) 


706(04) 


59 J (06) 


BRITISH COLUMBIA 


634(06) 


64 7 (06) 


773 (OS) 


IRELAND 


67 9(10) 


690(06) 


564(06) 


(RELANO 


553(09) 


481 (06) 


723(0 7) 


KOREA 


792(06) 


800(041 


72 3 (06) 


KOREA 


71 2 (0 7) 


74 7 (0 7) 


739(06) 


NEW BRUNSWICK (ENGLISH) 


732(06} 


71 1 (OS) 


580(06) 


NEW BRUNSWICK (ENGLISH) 


64 9(07) 


603 <08t 


736(0 6) 


NEW BRUNSWICK (FRENCH) 


729(08} 


692 (07| 


526(1 1) 


NEW BRUNSWICK (FRENCH) 


693(10) 


624(09) 


665(06) 


ONTARIO (ENGLISH) 


70 1 (0.7) 


677(06) 


56.1 (08) 


Ontario (English) 


564(09) 


59 7 (0.9) 


735(06) 


ONTARIO (FRENCH} 


627 (0 7) 


692(06) 


4? 4 (09) 


Ontario (frenCh) 


52 1 (0 61 


502(0 7) 


53 7(06) 


QUEBEC ^ENGLISH) 


75 9(05) 


723 (05) 


594 (04) 


QUEBEC (ENGLISH) 


628(0 7] 


62.3 (OS) 


74.1 (0.4) 


OUEBEC {FRENCH) 


779(07) 


7S6(06) 


609(08) 


OUEBEC (FRENCH) 


65 1 (0 &) 


621 e03) 


739(0 7] 


SPAIN 


686(10} 


706(10) 


62 7 (1 21 


SPAIN 


594(1 21 


5*5 (i3t 


723(0 9) 


UNITED KINGDOM 


615(09) 


736 (07J 


630(0.8) 


UNITED KINGDOM 


56 0(06) 


623(091 


7Bl (051 


UNITED STATES 


614 [10} 


599(1 1) 


491 (09) 


UNIT Ed STATES 


43.9(1 1) 


54 7(12) 


630(1 1) 
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Mathematics Mean OppQrtunity-to -Learn Ratings and Jackknifed Standard Errors by Topic 





NUMBERS 


RELATIONS, 








DAI A 


LDD1C AMD 




AND 


FUNCTIONS. 






MEASURE* 


ORGAN. 


PROBLEM 




OPERATIONS 


ETC. 


GEOttiTHlV 




WENT 


TATICW. ETC. 


SOLVING 


BftlTJSK COLUMBIA 


66 3 [1 6) 


597(1.5) 


306 (2 2h 


BRITISH COLUMSlA 


SS6 (2 2l 


43 J> f2 2h 


4ia,t9i 


IRELAND 


62SM 9) 


605 {1 9) 


23 3(20) 


IRELAND 


S3 7 (2 2J 


36 3 f23j 


i?7fT9i 


KOREA 


67 2 f?4j 


73 5(20) 


67 7(23) 


KOREA 


729 (21) 


59 6 [2 cJ 


51 ? [2 7i 


NEW BRUNSWJCK 1 ENGLISH) 


SB? L1.S) 


S?2(06i 


3?0[l4j 


new Brunswick (English ) 


U.3) 


512 (20i 


323(1 8j 


NEW BRUnSWJCK [FRENCH] 


GO 4 (23) 


730(26) 




NEW BRUNSWICK (FRENChI 


77$ |2 7j 


501 mjj 




Ontario English ) 


723 [T9> 


669 (?3i 


409)301 


ONTARIO (ENGLISH) 


692 (2 3} 


60 2 0 1 h 




0N7ARS0 {FRENCH) 


7<2[14) 


66 6 Li 6) 


46 6 12 1 ) 


ONUfltO (FRENCH 1 


692 <i 1\ 


62 3 <2 1 h 


550 4211 


QUE6EC (ENGL EH) 


73* (1 1) 


635 {1 3i 


3t 6 (1.7) 


QUE SEC (ENGLISH) 


592 (14) 


532 <:c> 


470 [1 1} 


QUEBEC (FRENCH t 


615 (1-S) 


?33(1 2> 


51 5(21} 


QUEBEC (FRENCH} 


67 1 [1.6] 


593 (25} 


599 (2 Ci 


SPAIN 


&1?[22) 


&3S(23> 


67.1 (39) 


SPAIN 


60/ (24) 


63 8 (3 7) 


56 6 (3.5J 


UNITED KINGDOM 


S?.? L?-7> 


662(211 


47 2(2 7} 


UNITED KINGDOM 


7m (21) 


55 5 i2.6i 


593(26j 


UNITED STATES 


690 120) 


62012 3) 


351 [36} 


UNITED STATES 


S9.i [3D) 


65 6 [3 1 ] 


42 4 (2 S) 



Science Proficiency Means and Jackknifed Standard Errors 





TOTAL 


MALE 


FEMALE 


BRITISH CQLUM9IA 


5Si.3(2U 


5623(321 


53,6(23) 


IRELAND 


3693 (551 


4503(521 


356 S (3 6) 


KOREA 


5499(29} 


56? 5 (36) 


5306(3 1} 


NEW BRUNSWICK (ENGLISH J 


SlOS(2 7) 


5-72(40) 


502.3 (4.1} 


NEW BRUNSWICK [FPcNCH) 


366 1 (39) 


377.4 


*602(4 3} 


ONTARIO (ENGLISH) 


SI 3.7 (2 71 


52*3(36) 


5047 (3.0J 


ON f ARi'J (FRENCH I 


see a (22) 


i?49 L 31| 


363 1 [2 7) 


QUEBEC {ENOLlSHI 


5153 {261 


525 J (361 


505.7 (31) 


QUEBEC (FRENCH! 


S134{33) 


523.7 (4.1} 


502 1 (3 6) 


SPAIN 


5039(4 31 


5]60(36) 


3895 (SO) 


UNITED KINGCDM 


5t95 (3.7J 


524 7(63) 


Sis9(33} 


UNHED STATES 


*7&.S [4 6) 


*81 9 (61) 


473 9(3 7} 



AMOUNTS OF WEEKLY sClEKCE HOMEWORK NUMBER OF SCIENCE ACTIVITIES WITH SOMEONE AT HOME 





LESS THAN 


1*2 


3 If OURS 




0 


1 


2 


3 




1 HOUR 


HOURS 


OR WDRE 




BRlTISh COLUMBIA 


5453(32) 


553 3(2.6) 


5717 (59) 


BRITISH COLUMBIA 


S37 5 (3.3) 


545.1 (2 7 


55*0 (40i 


SCI 3 i3 5> 


IRELAND 


462 J (37) 


495 7 |3 7} 


■193 7 


IRELAND 


452 5(36} 


4606(3 9) 


3601 [42i 


5016 iE Oh 


KOREA 


545.5 {3 3) 


SSi.7 (3.6} 


S&4 3 (76) 


KOREA 


5192 (5 7} 


5360(3 5) 


J>59Q(3i] 


5546(3 5) 


NEW BRUNSWICK (ENGLISH » 


5103(35) 


5116(51} 


503 3 (63} 


NEW BRUNSWICK (ESGUGM) 


496 5(4 6} 


5930(3 11 


5243 US 5 1 


5?70j? 1} 


NEW BRUNSWICK (FRENCH) 


3&S5{<2| 


463 9(3 9} 


3763 (6 5h 


NEW BRUNSWICK (FRENCH) 


4563(52} 


461 7(3 71 


3726 (5 9j 


3390 lS4h 


Ontario (English* 


5166(29) 


Si23IS0) 


5055 (H 3) 


Ontario (English* 


507 7 (3 6) 


Sl09<3 3) 


5166(311 


5306(6 if 


ONTARIO (FRENCH) 


471 <J 42 


3656(33) 


<66 3 [100) 


ONTARIO (FRENCH) 


4609 (3 4> 


472 3(3 21 


473Q|30] 


4728 E5 Oi 


QUEBEC (ENGLISH} 


St35 (31; 


ill 2(36) 


5592 i?0} 


QUEBEC (ENGLISH) 


3968(4 7} 


S1S.2 (33) 


&19B137) 


537613^ 


Quebec (frenCh) 


512.7 (3S) 


5165(3 3) 


5182 (03} 


QUEBEC [FRENCH) 


S1 1. 8{<.0t 


503 7 (3 2) 


5l52i47i 


5279,3 7) 


SPAIN 


3999(60] 


5003 (3 5) 


5222 (56) 


SPAIN 


5032 (5 3} 


6053(60] 


506 Mi 6 1 


5062 i6 3> 


UNITED K1HG0QM 


4999 (3 7) 


547.3(3 1) 


5655 (9^ 


UNITED KINGDOM 


5073 (3 6} 


5146(3 3) 


515 3[ t - :•! 


5J>9 /1M} 


UNITED STATES 


47AS (56) 


4&ISI7 5) 


4723 [76) 


UNITED STATES 


4604(6 51 


3605 (7 51 


36A j [7?J 


3^7 7^3) 



AMOUNTS OF DAILY TELEVISION VIEWING 





0*2 


3-4 


5 HOURS 




HOURS 


HOURS 


OR MDRF 


BRITISH COLUMBIA 


569 6(301 


5379 (2 7\ 


Sl93^ 2) 


IRELAND 


3652 (3 7} 


4655(3 2) 


3?36i60) 


KOREA 


5623 (39} 


Sfl3i (31) 


5179(7 1) 


NEW GRUNSWiCK (ENGLl5Hh 


531 4(3 7} 


504 & (fir; 


3930(60) 


NEW BPUNSWtCK (FRENCH} 


J790(3S} 


467 * «3 0} 


jS3SiG6> 


ONTARIO [ENGLlSHn 


5319 43 5) 


'►3 5136) 


<% 0 [3 9) 


ONTARIO (FRENCH) 


3622(3 1) 


3692 (2fl> 


346.6 (4 6) 


OUEBEC {ENGLISH) 


530 6 <4 C „ 


5126(3^) 


3?l 1 (3 6) 


QUEBEC (FRENCH) 


522^ [3 6} 


5090(31} 


390 7 (6 0] 


SPAIN 


^}4S[5 6} 


500 5(3 5} 


3776(63) 


UNITED hINGDOW 


537 ? f5l) 


S3i 1 (39} 


383 2 15 3) 


UNITED STATES 


3951 (7 4} 


466 \ \i 4} 


4535 |6 7) 
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APPENDIX 



Percentage of Students with Science Proficiency 
At or Above Each Level and Jackknifed Standard Errors 





LEVEL 300 


LEVEL 400 


LEVEL 50Q 


lSVEL 600 


LEVEL TOO 


BRITISH COLUMBIA 


997 |Ql) 


OS ^ 10SI 


71041 1) 


306410? 


41 [Oil 


IRELAND 


95 7 I99j 


75 6 41 1) 


37 241 &} 


01 409? 


06(02) 


KOREA 


99 7 102) 


03l 1071 


727 41 4) 


32641 3r 


23 (OA) 


NE'.V BRUNSWICK 1 ENGLISH 1 


90 2 102) 


90 A 40 7? 


55*11 5) 


149409? 


10 [03? 


NEW BRUNSWICK iFREnChJ 


95 0 jOi) 


779416? 


353(1 ?l 


45 3 4O0h 


QJ (02) 


ONTARIO (ENGLISH? 


9.35 40 3? 


90 6 40 Oh 


55011 3) 


171 4 "3 1 h 


1 G [03? 


ONTARIO [FRENCH? 


9B0 40 3? 


7& 3 40 Oh 


348<1 1| 


61 i06> 


02 ion 


QUEBEC (ENGLISH} 


095 402? 


01 6(0 7? 


574 <1 7? 


14 a 40 en 


l 5 i03> 


QUEBEC [FRENCH J 


09 6 10 1 > 


9l 5(09} 


563(1 7? 


1^241 3? 


08 [02l 


SPAtN 


901 i0 2) 


eao n.i) 


535 42 31 


122 Li 3) 


06(02) 


united Kingdom 


98 5(Q3i 


000 (1 0) 


590fl 3J 


23 3 ri A) 


2A (041 


UNITED STATES 


95&IO&) 


F83 11 9) 


*1 8126) 


118 (l it 


1 A £0 4) 



Science Mean Perron ts Correct and Jackknifed Standard Errors by Topic 











EARTH 


NATURE 




LIFE 






m SPADE 


OF 




SCIENCES 


PHVSICS 


Chemistry 


SCIENCES 


SCIENCE 


British Columbia 


726 lQ<p 


63 7 (Cat 


6a a ro 5? 


730 [04) 


603 40 4? 


IRELAND 


600 106? 


530(05? 


7 (0 SJ 


*>i 0 (Oft) 


5*5108? 


KOR£A 


72 7 10 5) 


676(05? 


659 (0 6} 


71 3 (05) 


653 406? 


fiEW BRUNSWfcK 1ENGLlSh+) 


£60 405? 


503(04? 


538 (06) 


662(06) 


63 0)0 5? 


NEW BRUNSWKK IFRfcNCHI 


586 dO 7h 


i&0(07? 


Jfl8ioa.i 


552 (0 7) 


57 3(08? 


ONTARIO (EnGLCSH} 


67 4 40 Sh 


598 (06J 


528 *0 6) 


68 0 (O 61 


639105? 


ONTARIO (FRENCH) 


901 405? 


SSI (04} 


;6 9 (0 Si 


57.3 (05) 


566 405? 


QUEBEC (ENGLISH) 


689 105? 


593 (OS? 


Si ,4 10 6] 


66 3 [05) 


64 4(06} 


QUEBEC [FRENCH J 


706(06? 


596 (06> 


54 010.7) 


60 7 [0 7) 


64 0(06? 


SPAIN 


60 0 4 E> 45 h 


60 2 (08} 


5i 6 (1 0) 


656 (1 2) 


59$ (03? 


UNITEQ KINGQQM 


406} 


622 [0 7} 


52 4 40 7) 


688 (08) 


64 2(0 7? 


UNITED STATES 


630 11 01 


529 [09} 


17 7 1l 0) 


Si 4 n i) 


560(1 Q} 



Science Mean Opportunity-to-Leam Ratings and Jackkni/ed Standard Errors by Topic 











EARTH 


NATURE 




LIFE 






AND SPACE 


UF 




StiEKCES 


PHYSICS 


CHEMISTRY 


SCtEHtES 


SCIENCE 


BRITISH COLU(Ll6lA 


23011 4? 


2t&H 7? 


52i L30J 


238 Ll 9) 


30312 2? 


IRELAND 


254 11 71 


25l [1 7j 


30 3 {2 6) 


160 (T9) 


370(21} 


KOREA 


35511 7} 


39 Ml 6? 


528(16) 


304 fl 6) 


3&< (l 7? 


NEW BRUNSWICK 4 ENGLISH? 


33/(191 


3l 1 i22} 


339 (1 7) 


334 (20) 


£46ll 6? 


NEW &FWNSWICK (FRENCH) 


300(3 ?} 


29B (35} 


46 6 (3 7) 


400 [36? 


31 6 (34? 


Ontario (English) 


399 [24) 


28641 3) 


39 0(26? 


363 423? 


460 a 2) 


ONTARIO (FRENCH! 


366 (2 0) 


263 HO! 


366(25? 


31.3(16? 


AA ^126) 


QUEBEC (ENGLISH) 


331 (1 3) 


1 1 0 |l li 


231 (l 4? 


262(1 2} 


33 3 11 7? 


QUEBEC (FRENCH.! 


420 [2l? 


15)9(1 3) 


25.9 (1 6} 


31 0 (1 6} 


37&118J 


SPAIN 


648 (33? 


572(361 


632(3 3? 


65 I (27) 


6l 7 42 7) 


UNITED KINGDOM 


360 123) 


322 [21? 


656(25? 


239 (2D 


399129? 


I'M TED STATES 


43 T 125? 


259(251 


Al 7 [25} 


403 (341 


42,1 129) 




Their Contributions Made A World of Differences 



Any international project is necessarily complex because of cultural and linguistic differences. 
Success depends on the sensitivity and dedication of the individuals involved, who must consis- 
tently make the extra effort to communicate the issues and to grasp important nuances. This study 
indeed was fortunate to attract a group of gifted and talented people who consistently exceeded the 
expectations of vigorous standards and demanding schedules. 

Solange PaqueL Leo Laroche, Denis Savard, and Paul Vachon from the Quebec Ministry of 
Education helped shape the early conception of the project and efficiently managed their 
province's activities during the entire effort. Their imagination, along with the enthusiastic 
support of Clare Burstal! from the National Foundation for Educational Research in Engird and 
Wales (NFER) energized the idea into reality. 

Wendy Keys, also from the NFER, coordinated the project for the United Kingdom with 
assistance from Derek Foxman, Douga! Hutchison, and Barbara Bloomfield. Scottish participation 
was made possible by SaHy Brown, Graham Thorpe, and Susan Freshwater of the Scottish Council 
for Research in Education, 



Mariano Alvaro of the Ministry of Education and Science in Spain along with his colleagues, 
[gnacio Gozanlo-Misol Maria Jose Navas, Susana Marcos, and Modesto Escobar joined the 
fledgling project and coordinated Spain's participation. Jean Jae Lee, Woong Sup Yoon, and Jin 
Gyu Kim from the National Institute of Educational Evaluation successfully directed the activities 
in Korea. Thomas Ketlaghan and Michael Martin from the Educational Research Center. St. 
Patricks College, organized and managed Ireland's project. 

In addition to Quebec where Allen Patenaude from the Ministry 1 of Education directed the 
English schools* participation, three other Canadian provinces successfully joined in the project, 
thanks to the consistent and effective efforts of Sylvio Chenard. Leo-Paul Charest. Guy l^eveitle* 
and Laurie Boucher from the New Brunswick Ministry of Education (Francophone): Richard 
Harvey and Gary Grobe, New Brunswick Ministry of Education (Angiophone); William Lipischak, 
Ron CussonSt Dennis Raphael, Jacqueline Fortin-LaCoste, and Francois Lavictoire from the 



ERLC 
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Ontario Ministry of Education; and Barry Carbol and Michael Kozlow from the British Columbia 
Ministry of Education. 

Early and consistent encouragement for the whole notion was provided by Dick Berry of the 
National Science Foundation, and Emerson Elliott, Acting Commissioner. National Center for 
Education Statistics (NCESh In addition, support was provided by Gary Phillips, Larry Suter, and 
Maureen Treacy from the NCES. 

The data analyses tor the pilot tests for various countries and development of the data base for 
the final assessments were completed by Leo Laroche, with the assistance of Nicole Dessureault 
and Nathalie Laroche. Data analysis was managed by Frangois Dupuis and Richard Bertrandat the 
University of Laval with the able and tireless assistance of Normand Dufourand Marc Letourneau. 

Research staff at ETS provided consultative help, especially Eugene Johnson, Bob Mislevy, John 
Barone, Bruce Kaplan, and Ed Kulick. Ben King and Jim Ferris, also at ETS, designed the sampling 
scheme and monitored its implementation in ail of the participating countries. 

Nancy Mead at ETS acted as project director with Marion Epstein providing able backup and 
advice, Solange Paquet at Educan coordinated the Canadian data-analysis activities. She received 
assistance in translation from Verna Delaney, 

Initial drafts of themanuscrtpt benefited from the technical reviews of Eugene Johnson and Ann 
Jungeblutat ETS, John Dossey of Illinois State University, Senta Raizen of the National Center for 
Improving Science Education, Lyle Jones of the University of North Carolina, and the Board on 
International Comparative Studies in Education directed by Dorothy Gilford. 

Georgia Connor and JoAnn Piazza at ETS deserve abundant credit for careful and repeated 
manuscript preparation throu + j multiple edits. Kent Ashworth and Jan Askew managed the 
important tasks of publication ^d dissemination with imagination and great good humor. Jack 
Weaver's creative design gave me final product its distinctive appearance. 
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What proportion of your country's 13-year-old students 
would you expect to be able to answer questions like these? 



MATHEMATICS 
LEVEL 600 




The length of a side of this square is 6, What is the radius of the circle? 
CD 2 CD 3 CD 4 CD 6 CD 8 CD 9 CD J don't know. 



SG1ENCE 

LEVEL 600 Group A Group B Group C 

Water vapor Ice Alcohol 

Oxygen Aluminum' Water 

Air Iron Gasoline 

The substances above, each at room temperature, have been classified into 
gToups. On what property is the classification based? 

CD Chemical composition 

CD Specific heat 

CD State of matter 

Abundance within the Earth's crust 



ERIC 
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